-- Logs begin at Mon 2022-10-24 12:31:34 UTC, end at Mon 2022-10-24 13:32:42 UTC. -- Oct 24 12:58:41.092093 dv-cinder-862489-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 24 12:58:41.099638 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 24 12:58:41 2022] *** Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: nodename: dv-cinder-862489-1-iscsi Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: machine: x86_64 Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: clock source: unix Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: pcre jit disabled Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: detected number of CPU cores: 4 Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: current working directory: / Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: detected binary path: /usr/bin/uwsgi-core Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: your processes number limit is 47680 Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: your memory page size is 4096 bytes Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: detected max file descriptor number: 2048 Oct 24 12:58:41.101779 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: lock engine: pthread robust mutexes Oct 24 12:58:41.102544 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: thunder lock: enabled Oct 24 12:58:41.102544 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 24 12:58:41.102544 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 24 12:58:41.141895 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: Python main interpreter initialized at 0x559c0166d9c0 Oct 24 12:58:41.141895 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: python threads support enabled Oct 24 12:58:41.141895 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: your server socket listen backlog is limited to 100 connections Oct 24 12:58:41.141895 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: your mercy for graceful operations on workers is 90 seconds Oct 24 12:58:41.142745 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: mapped 403077 bytes (393 KB) for 2 cores Oct 24 12:58:41.142745 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: *** Operational MODE: preforking *** Oct 24 12:58:41.142745 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: *** uWSGI is running in multiple interpreter mode *** Oct 24 12:58:41.142745 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: spawned uWSGI master process (pid: 81599) Oct 24 12:58:41.142815 dv-cinder-862489-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 24 12:58:41.143945 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: spawned uWSGI worker 1 (pid: 81613, cores: 1) Oct 24 12:58:41.143945 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: spawned uWSGI worker 2 (pid: 81614, cores: 1) Oct 24 12:58:41.143945 dv-cinder-862489-1-iscsi devstack@keystone.service[81599]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 24 12:58:45.440615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 12:58:45.440615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: from cryptography.utils import int_from_bytes Oct 24 12:58:45.440615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 12:58:45.440615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: from cryptography.utils import int_from_bytes Oct 24 12:58:45.441827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 12:58:45.441827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: from cryptography.utils import int_from_bytes Oct 24 12:58:45.442247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 24 12:58:45.442247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: from cryptography.utils import int_from_bytes Oct 24 12:58:45.490466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 12:58:45.490651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 12:58:45.490832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] command line args: [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 12:58:45.491126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 12:58:45.491310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ================================================================================ {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 12:58:45.491566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] admin_token = **** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.491811 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] config_dir = [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.492000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.492184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] config_source = [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.492365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] debug = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.492565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.493726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.493726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.493726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.493726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.494174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] list_limit = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.494174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] log_config_append = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.494174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.494443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] log_dir = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.494521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] log_file = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.494703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.494909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 24 12:58:45.495004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.495116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 24 12:58:45.495197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.495305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] command line args: [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 24 12:58:45.495447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.495576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 24 12:58:45.495659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.495742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ================================================================================ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 24 12:58:45.495818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.495993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] admin_token = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_dir = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_source = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] debug = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.496993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.497139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.497219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.497346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.497433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.498172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.498351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.498518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.498648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.498905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_config_append = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] publish_errors = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_dir = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499633 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499823 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.499898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] standard_threads = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.500884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] use_journal = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501430 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] use_json = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] use_stderr = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.501911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] use_syslog = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.502060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.502152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.502294 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.502376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.502543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.502618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.502789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.502865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.503044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.503157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.503323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.503563 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.503645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.503772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] publish_errors = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.503851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.503973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.504048 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.504207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.504298 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.504378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.504509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.504621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.504765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.504847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.505004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.505101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] standard_threads = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.505253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.505337 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.505492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.505567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.505692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.505783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.505855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_journal = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.505986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.506139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_json = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.506214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_stderr = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.506283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.506361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_syslog = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.506719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.506833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 24 12:58:45.506977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.507085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.507252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.507343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.507491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.507570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.507746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.507826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.508976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.509059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.509206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.509283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.509443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.509606 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.509681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.509750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.509926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.510957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.511132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.511214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.511385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.511474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.511619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.511697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.511855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.511938 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.512092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.512169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.512312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.512393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.512544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.512626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.512775 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.512852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.513008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.513086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.513292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.513386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.513543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.513620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.513783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.513862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.514043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.514120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.514292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.514372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.514526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.514604 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.514761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.514837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515582 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.515978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.516988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.517286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.517413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.517523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.517620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.517620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.517806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.517902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518506 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.518966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.519132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.519245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.519354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.519507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.519585 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.519696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.519824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.519946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.520974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.521120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.521198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.521360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.521444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.521596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.521724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.521815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.521887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.522070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.522156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.522305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.522393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.522535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.522612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.522771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.522848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.523005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.523133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.523295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.523375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.523536 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.523613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.523785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.523870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.524975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.525993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.526139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.526223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.526370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.526446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.526604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.526752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.526826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.526895 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.527083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.527163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.527324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.527410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.527581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.527657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.527821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.527896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.528072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.528153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.528305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] auth.external = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.528383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.528538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.528614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.528804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.531725 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.531840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.531918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.531990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] auth.password = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] auth.token = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.532957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] credential.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533552 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.533981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.external = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534604 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.password = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.534984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535075 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.token = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.535988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] federation.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536470 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536470 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.536972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.537949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.538027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.538195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.538281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.538444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.538523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.538680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.538757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.538916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539188 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.539986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.540156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.540232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.540386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.540537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.540611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.540681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.540835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.540913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.541078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.541163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.541325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.541403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.541572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.541649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.541827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.541961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.542065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.542142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.542299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.542375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.542549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.542634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.542783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.542859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.543978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.544062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.544216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.544980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.545191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.545268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.545268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.545406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.545479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.545637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.545716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.545897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.546966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.547138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.547226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.547371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.547450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.547611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.547690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.547843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.547920 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.548083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.548193 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.548315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.548438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.548550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.548678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.548790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.548932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549650 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.549981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.550153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.550233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.550391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.550470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.550629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.550707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.550867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.550945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.551125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.551204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.551363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.551440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.551609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.551686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.551840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.551919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.552080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.552156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.552321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.552399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.552561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.552641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.552795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.552878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.553959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.554038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.554199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.554275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.554434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.554512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.554683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.554761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.554932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.555010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.555211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.555291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.555446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.555525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.555695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.555773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.555934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.556960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.557121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.557198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.557357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.557433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.557594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.557672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.557828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.557905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.558069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.558146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.558306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.558385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.558540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.558618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.558779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.558856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.559025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.559120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.559280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.559359 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.559519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.559602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.559757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.559835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.560043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.560728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] resource.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.560803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.560951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.561076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.561201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.561280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.561441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.561519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.561693 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.561778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.561944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562257 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562668 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.562980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.563163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] role.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.563257 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.563413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.563489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] role.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.563642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.563718 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.563882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.563967 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.564123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.564200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.564365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.564442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.564613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.564693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.564863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.564939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.565110 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.565196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.565359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.565436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.565606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.565682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.565838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.565916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.566085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.566162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.566326 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.566403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.566567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.566644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.566803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.566888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.567072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.567234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.567322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.567473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.567559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.567725 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.567804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.567975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.568053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.568225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.568315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.568472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.568551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.568706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.568785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.568958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569906 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.569984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.570138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.570215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.570391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.570470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.570646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.570785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.570893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.571025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.571161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.571316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.571394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.571573 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.571650 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.571820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] token.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.571904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.572059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.572144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.572295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.572372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.572530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.572614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.572785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.572862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.573969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.574993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.575173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.575252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.575413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.575502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.575744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.575850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.connection = **** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.576989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.577066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.577229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.577309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.577463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.577540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.577700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.577778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.577935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578406 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578718 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.578960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.579147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.579229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.579399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.579487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.579636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.579716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.579880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.579957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.580110 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.580187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.580348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.580427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.580582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.580660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.580752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81614) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 12:58:45.580830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.581066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.581310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.581549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.581788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.582034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.582272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.582517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.582752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.583000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.583261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.583507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.583739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.583980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.584216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 24 12:58:45.584386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 24 12:58:45.610324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.610553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.610778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.613117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.613355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.613584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.619091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.621850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.649246 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.649481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.649566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.649790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.702186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.702319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.751024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.751282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.751556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.751793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.907633 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.907820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.907936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.908088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.908230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.908381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.908512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.908639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.908778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.908910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.909045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.909239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.931470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.931705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.931843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.931976 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.932119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.932247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.932384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.932510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.932634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.932766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.932891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.933015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.933140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.933140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.944213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.944663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.957558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.957737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.957868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.958009 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.958135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.958260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.958383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.958507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.958630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.958762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.958886 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.959023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.959173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.959303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81614) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 24 12:58:45.975426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.975638 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 12:58:45.975981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.976108 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.976237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 12:58:45.976366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 12:58:45.976506 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.976634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.976760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.976899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 24 12:58:45.977027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.977201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.977326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.977454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.977784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.978055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.986899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.987213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.987491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.987617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.987782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.987909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.988134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.988423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.994718 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:45.995350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:45.998167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:45.998457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.998746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.998869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:45.999028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.999191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.999319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.999456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.999591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.999718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.999844 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:45.999969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.000103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.000231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.000402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.000525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.000662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.000796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.001015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.001139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.001320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.001594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.024306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.024926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.025777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.026068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.026390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.026681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.032468 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.032759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.033045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.033169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.033389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.033681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.039511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.039805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.040070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.040362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.043960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.044260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.045042 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.045171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.045344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.045472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.045634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.045759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.045933 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.046057 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.046215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.046366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.046489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.046612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.046752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.047035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.062129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.062414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.062556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.062848 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.066264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.066710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.069490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.069795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.069919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.070220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.074010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.074409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.077781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.078123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.081023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.081329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.081414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.081619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.081701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.081919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.082010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.082216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.082299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.082499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.082580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.082789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.098802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.099119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.099204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.099396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.099496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.099668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.099751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.099933 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.106498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.106740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.108317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.108463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.109844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.109936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.110130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.110212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.112764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.112847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.113057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.113139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.113450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.113551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.113740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.113824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.114025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.114109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.125084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.125344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.125427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.125636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.125743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.126018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.126109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.126306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.126389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.126605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.126694 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.126889 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.126970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.127211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.142888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.143435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.145786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.146338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.148657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.149251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.150176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.150715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.151808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.152109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.152393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.152528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.152761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.153051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.159237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.160065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.160709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.161526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.163718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.164017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.164308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.164517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.164656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.164797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.165092 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.165378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.172559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.172859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.173150 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.173281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.173488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.173776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.180898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81614) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.181191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.181477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.181606 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 24 12:58:46.181758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.181884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.182023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.182160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.182287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.182415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.182566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.182690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.182837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.182972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.183147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.183271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.183404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.183529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.183679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.183804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.183952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.184085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.184229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.184379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.184507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81614) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.184629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.184800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.185099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 24 12:58:46.218336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559c0166d9c0 pid: 81614 (default app) Oct 24 12:58:46.218806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x559c0166d9c0 pid: 81613 (default app) Oct 24 12:58:46.221658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cf3e-d2e5-4189-bf7e-757b04301f6f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:46.221952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cf3e-d2e5-4189-bf7e-757b04301f6f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:46.222230 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d0cf3e-d2e5-4189-bf7e-757b04301f6f None None] PATH_INFO: `/v3` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:46.226409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Mon Oct 24 12:58:46 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:58:53.040095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d3d3c-d4c7-4cf5-bb9a-c495d8233d35 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:53.040607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d3d3c-d4c7-4cf5-bb9a-c495d8233d35 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:53.040713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d3d3c-d4c7-4cf5-bb9a-c495d8233d35 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:53.044979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:58:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:58:53.050361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:53.050799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:53.050940 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:53.121091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: INFO dbcounter [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] Registered counter for database keystone Oct 24 12:58:53.140874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG oslo_db.sqlalchemy.engines [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81614) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 12:58:53.147012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writer thread running {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 12:58:53.322911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG passlib.handlers.bcrypt [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81614) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 24 12:58:53.323239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG passlib.handlers.bcrypt [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81614) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 24 12:58:53.381260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:58:53.383164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:58:53.508905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:58:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 12:58:53.974722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d3d3c-d4c7-4cf5-bb9a-c495d8233d35 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:54.043960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: INFO dbcounter [None req-674d3d3c-d4c7-4cf5-bb9a-c495d8233d35 None None] Registered counter for database keystone Oct 24 12:58:54.065431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG oslo_db.sqlalchemy.engines [None req-674d3d3c-d4c7-4cf5-bb9a-c495d8233d35 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81613) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 24 12:58:54.071775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writer thread running {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 24 12:58:54.189297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d3d3c-d4c7-4cf5-bb9a-c495d8233d35 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:54.193627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3180d5-0484-4609-8e73-d75b097e32eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:58:54.194252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3180d5-0484-4609-8e73-d75b097e32eb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:54.194536 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3180d5-0484-4609-8e73-d75b097e32eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:54.194811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3180d5-0484-4609-8e73-d75b097e32eb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:54.203485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3180d5-0484-4609-8e73-d75b097e32eb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:58:54.254514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9a3180d5-0484-4609-8e73-d75b097e32eb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 24 12:58:54.254810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3180d5-0484-4609-8e73-d75b097e32eb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:58:54.264492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Oct 24 12:58:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 12:58:54.270569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:54.287710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecc4196-6a7a-4cee-9c03-ea4c5edd44df None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:54.292068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54480fc-eb86-4403-b835-1cf5e1eab699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:58:54.293171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54480fc-eb86-4403-b835-1cf5e1eab699 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:54.293452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54480fc-eb86-4403-b835-1cf5e1eab699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:54.293721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54480fc-eb86-4403-b835-1cf5e1eab699 admin admin] PATH_INFO: `/v3/projects/f3929898cf764cd6a05e8662e83d95e0` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:54.298055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54480fc-eb86-4403-b835-1cf5e1eab699 admin admin] RBAC: Authorizing `identity:get_project(project_id=f3929898cf764cd6a05e8662e83d95e0)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:58:54.348440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d54480fc-eb86-4403-b835-1cf5e1eab699 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 24 12:58:54.348780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54480fc-eb86-4403-b835-1cf5e1eab699 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:58:54.353229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 24 12:58:54 2022] GET /identity/v3/projects/f3929898cf764cd6a05e8662e83d95e0 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:58:56.471311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041cf9-338b-437a-8a6d-27490c784a70 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:56.471821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041cf9-338b-437a-8a6d-27490c784a70 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:56.471903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87041cf9-338b-437a-8a6d-27490c784a70 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:56.475964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:58:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:58:56.482237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1447-17b3-4ce3-adef-97e700cd324a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:56.482752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1447-17b3-4ce3-adef-97e700cd324a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:56.483045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e1447-17b3-4ce3-adef-97e700cd324a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:56.503279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c30e1447-17b3-4ce3-adef-97e700cd324a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:58:56.504788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c30e1447-17b3-4ce3-adef-97e700cd324a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:58:56.512949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:58:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 12:58:56.786754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87041cf9-338b-437a-8a6d-27490c784a70 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:56.799142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87041cf9-338b-437a-8a6d-27490c784a70 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:56.803370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110b9ee-200e-4308-a7b6-69c781e7ff47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:58:56.803974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b9ee-200e-4308-a7b6-69c781e7ff47 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:56.804261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b9ee-200e-4308-a7b6-69c781e7ff47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:56.804541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b9ee-200e-4308-a7b6-69c781e7ff47 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:56.812609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110b9ee-200e-4308-a7b6-69c781e7ff47 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:58:56.814556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110b9ee-200e-4308-a7b6-69c781e7ff47 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:58:56.825945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Mon Oct 24 12:58:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 12:58:56.831673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1447-17b3-4ce3-adef-97e700cd324a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:56.840734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e1447-17b3-4ce3-adef-97e700cd324a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:56.845910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d217c3-b9eb-474e-aa14-b5aa1f6ef353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:58:56.846561 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d217c3-b9eb-474e-aa14-b5aa1f6ef353 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:56.846845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d217c3-b9eb-474e-aa14-b5aa1f6ef353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:56.847154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d217c3-b9eb-474e-aa14-b5aa1f6ef353 admin admin] PATH_INFO: `/v3/users/c47c715f72104fe1bd8f50179af11065` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:56.848463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d217c3-b9eb-474e-aa14-b5aa1f6ef353 admin admin] RBAC: Authorizing `identity:get_user(user_id=c47c715f72104fe1bd8f50179af11065)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:58:56.850285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d217c3-b9eb-474e-aa14-b5aa1f6ef353 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:58:56.851828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Mon Oct 24 12:58:56 2022] GET /identity/v3/users/c47c715f72104fe1bd8f50179af11065 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:58:59.161924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f17db-9868-4f0a-912f-f67b9325a9fb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:59.162457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f17db-9868-4f0a-912f-f67b9325a9fb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:59.162457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f17db-9868-4f0a-912f-f67b9325a9fb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:59.163828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:58:59.169042 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8795a-4403-4eca-8333-8be6de9db44f None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:59.169982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8795a-4403-4eca-8333-8be6de9db44f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:59.170334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8795a-4403-4eca-8333-8be6de9db44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:59.189146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-49b8795a-4403-4eca-8333-8be6de9db44f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:58:59.190308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-49b8795a-4403-4eca-8333-8be6de9db44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:58:59.192537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb626a7-cf30-4209-ae9c-687c9e342337 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:59.192892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb626a7-cf30-4209-ae9c-687c9e342337 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:59.193171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb626a7-cf30-4209-ae9c-687c9e342337 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:59.194504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:58:59.199709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:59.200064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:59.201031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:59.242168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG passlib.handlers.bcrypt [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81613) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 24 12:58:59.242558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG passlib.handlers.bcrypt [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81613) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 24 12:58:59.275737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:58:59.292613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:58:59.294248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:58:59.393333 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:58:59.546807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8795a-4403-4eca-8333-8be6de9db44f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:59.572548 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:58:59.654945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b8795a-4403-4eca-8333-8be6de9db44f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:59.662696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:59.688733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb94a1-73c3-4f2d-a619-60012e4713bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:58:59.689439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb94a1-73c3-4f2d-a619-60012e4713bf None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:59.689728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb94a1-73c3-4f2d-a619-60012e4713bf None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:59.690003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb94a1-73c3-4f2d-a619-60012e4713bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:59.701428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deb94a1-73c3-4f2d-a619-60012e4713bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-0deb94a1-73c3-4f2d-a619-60012e4713bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:58:59.710276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:58:59.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:58:59.712078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 12:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:58:59.717829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb94a1-73c3-4f2d-a619-60012e4713bf None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:59.763965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9fde82-17db-4d95-9158-5608bef0f5be None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:59.766336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:58:59.800013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:58:59.802524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:58:59.803697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:59.804053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:59.804394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:59.805372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:58:59.806818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:58:59.817591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.sql.core [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 12:58:59.817591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 12:58:59.817591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [parameters: {'id': '057e16a1c1e341f391b58405fe083ee0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 12:58:59.817591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81613) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 12:58:59.818416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb94a1-73c3-4f2d-a619-60012e4713bf None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 12:58:59.836692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 12:58:59.837627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application yield resource Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise self.value Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application session.commit() Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 12:58:59.838698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.flush() Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application compat.raise_( Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 12:58:59.839935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 12:58:59.841610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 12:58:59.843084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [parameters: {'id': '057e16a1c1e341f391b58405fe083ee0', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 12:58:59.844420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:58:59.845361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 8/15] 10.222.0.42 () {60 vars in 1137 bytes} [Mon Oct 24 12:58:59 2022] POST /identity/v3/domains => generated 190 bytes in 176 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:58:59.846457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:59.859467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:58:59.860089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:59.860386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:59.860667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:58:59.861333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:58:59.861636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:58:59.863408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:58:59.874988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 8/16] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:58:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:58:59.880417 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:58:59.947836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f6b91-29d4-4256-9bbb-ae1fafe45819 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:59.977659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb203b-a1a0-48a0-820a-87b1f63a20de None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:58:59.980857 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:58:59.981714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:58:59.982001 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:58:59.982282 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:00.000502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:00.001959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:00.011512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ffaea-0f8f-4bfb-be8a-bb89a44b52eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:00.012165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ffaea-0f8f-4bfb-be8a-bb89a44b52eb None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:00.012466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ffaea-0f8f-4bfb-be8a-bb89a44b52eb None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:00.013315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ffaea-0f8f-4bfb-be8a-bb89a44b52eb None admin] PATH_INFO: `/v3/users/c47c715f72104fe1bd8f50179af11065` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:00.014718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ffaea-0f8f-4bfb-be8a-bb89a44b52eb None admin] RBAC: Authorizing `identity:get_user(user_id=c47c715f72104fe1bd8f50179af11065)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:00.016232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ffaea-0f8f-4bfb-be8a-bb89a44b52eb None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:00.017985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:58:59 2022] GET /identity/v3/users/c47c715f72104fe1bd8f50179af11065 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 12:59:00.021511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:00.022439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 12:59:00.023286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 12:59:00.023286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:00.023286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:00.023286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 9/18] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:00.023490 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ffaea-0f8f-4bfb-be8a-bb89a44b52eb None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:00.028800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:00.040433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:00.051325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:00.131047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b508e4-b56b-478a-8ff1-21c5fa86b0bd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:00.131647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ffaea-0f8f-4bfb-be8a-bb89a44b52eb None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:00.164314 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b130ff-64c6-4d9e-bba3-1ac8dafcb357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:00.164990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b130ff-64c6-4d9e-bba3-1ac8dafcb357 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:00.165156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f40b70-7f3b-44bf-9f9e-3271d3967303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:00.165268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b130ff-64c6-4d9e-bba3-1ac8dafcb357 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:00.165541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b130ff-64c6-4d9e-bba3-1ac8dafcb357 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:00.165779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f40b70-7f3b-44bf-9f9e-3271d3967303 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:00.166066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f40b70-7f3b-44bf-9f9e-3271d3967303 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:00.166242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b130ff-64c6-4d9e-bba3-1ac8dafcb357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:00.166346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f40b70-7f3b-44bf-9f9e-3271d3967303 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:00.166557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b130ff-64c6-4d9e-bba3-1ac8dafcb357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:00.167496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f40b70-7f3b-44bf-9f9e-3271d3967303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:00.167978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b130ff-64c6-4d9e-bba3-1ac8dafcb357 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:00.169592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f40b70-7f3b-44bf-9f9e-3271d3967303 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:00.171385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 10/19] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:00.176919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f40b70-7f3b-44bf-9f9e-3271d3967303 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:00.186930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 10/20] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 12:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:00.264716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:00.285189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f40b70-7f3b-44bf-9f9e-3271d3967303 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:00.321754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36aebf4-c4d1-49d9-b3a9-fca7e7902e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:00.322433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aebf4-c4d1-49d9-b3a9-fca7e7902e73 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:00.322725 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aebf4-c4d1-49d9-b3a9-fca7e7902e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:00.323013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36aebf4-c4d1-49d9-b3a9-fca7e7902e73 None admin] PATH_INFO: `/v3/domains/default/users/c47c715f72104fe1bd8f50179af11065/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:00.325094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36aebf4-c4d1-49d9-b3a9-fca7e7902e73 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c47c715f72104fe1bd8f50179af11065, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:00.326586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36aebf4-c4d1-49d9-b3a9-fca7e7902e73 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:00.446891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 11/21] 10.222.0.42 () {58 vars in 1361 bytes} [Mon Oct 24 12:59:00 2022] PUT /identity/v3/domains/default/users/c47c715f72104fe1bd8f50179af11065/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 12:59:02.855148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c02af-0c85-498d-92b3-8b8d57300c81 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:02.855674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c02af-0c85-498d-92b3-8b8d57300c81 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:02.855997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c02af-0c85-498d-92b3-8b8d57300c81 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:02.857778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 11/22] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:02.864417 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f91af2-c887-4978-8177-cf699e9abe35 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:02.865289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f91af2-c887-4978-8177-cf699e9abe35 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:02.865720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f91af2-c887-4978-8177-cf699e9abe35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:02.893991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-38f91af2-c887-4978-8177-cf699e9abe35 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:02.895828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-38f91af2-c887-4978-8177-cf699e9abe35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:02.933701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10faed-4b1b-44c1-a327-db579184d5d3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:02.934125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10faed-4b1b-44c1-a327-db579184d5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:02.935224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10faed-4b1b-44c1-a327-db579184d5d3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:02.937123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 12/23] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:02.943908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba978c2d-4442-4913-bfc6-95e766e06402 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:02.944309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba978c2d-4442-4913-bfc6-95e766e06402 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:02.944713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba978c2d-4442-4913-bfc6-95e766e06402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:02.984156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ba978c2d-4442-4913-bfc6-95e766e06402 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:02.985794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ba978c2d-4442-4913-bfc6-95e766e06402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:03.025316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 12/24] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:03.071783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dba87-5f2e-4252-8318-cad2c1bced12 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:03.072091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dba87-5f2e-4252-8318-cad2c1bced12 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:03.072348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8dba87-5f2e-4252-8318-cad2c1bced12 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:03.076655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 13/25] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:03.085671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cbd47-19f7-4ef9-a2fd-47f01b3b8dfb None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:03.085959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cbd47-19f7-4ef9-a2fd-47f01b3b8dfb None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:03.086244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cbd47-19f7-4ef9-a2fd-47f01b3b8dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:03.138831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e26cbd47-19f7-4ef9-a2fd-47f01b3b8dfb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:03.145665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e26cbd47-19f7-4ef9-a2fd-47f01b3b8dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:03.149811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 13/26] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:03.247523 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:03.255165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 14/27] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:03.414555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba978c2d-4442-4913-bfc6-95e766e06402 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:03.427868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:03.476919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76cae-7e40-469e-a28d-79a07dc03e82 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:03.477220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76cae-7e40-469e-a28d-79a07dc03e82 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:03.477493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76cae-7e40-469e-a28d-79a07dc03e82 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:03.479121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 15/28] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:03.486781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687f1a-093f-4317-b842-7418aba08671 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:03.487079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687f1a-093f-4317-b842-7418aba08671 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:03.487379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687f1a-093f-4317-b842-7418aba08671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:03.516674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-25687f1a-093f-4317-b842-7418aba08671 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:03.518763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-25687f1a-093f-4317-b842-7418aba08671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:03.577127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba978c2d-4442-4913-bfc6-95e766e06402 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:03.619463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee040135-8a12-4175-9ead-246d1a26d4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:03.620093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee040135-8a12-4175-9ead-246d1a26d4ba None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:03.620383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee040135-8a12-4175-9ead-246d1a26d4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:03.620662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee040135-8a12-4175-9ead-246d1a26d4ba None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:03.621349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee040135-8a12-4175-9ead-246d1a26d4ba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:03.622908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee040135-8a12-4175-9ead-246d1a26d4ba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:03.690252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 16/29] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 24 12:59:03.691145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687f1a-093f-4317-b842-7418aba08671 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:03.780317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 14/30] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 24 12:59:03 2022] POST /identity/v3/roles => generated 219 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:03.781547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee040135-8a12-4175-9ead-246d1a26d4ba None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:03.881820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687f1a-093f-4317-b842-7418aba08671 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:03.909372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:03.964462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:03.976166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:03.976834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:03.977126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:03.977405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:03.995704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee040135-8a12-4175-9ead-246d1a26d4ba None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:04.008149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:04.009998 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 12:59:04.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:04.049115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 12:59:04.049993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 12:59:04.049993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:04.049993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:04.049993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 17/31] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:04.050648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:04.085946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53adfb-c7af-4f18-ada0-b6f341464bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:04.086618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53adfb-c7af-4f18-ada0-b6f341464bfd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.086905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53adfb-c7af-4f18-ada0-b6f341464bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.087221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53adfb-c7af-4f18-ada0-b6f341464bfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.112045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53adfb-c7af-4f18-ada0-b6f341464bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-fb53adfb-c7af-4f18-ada0-b6f341464bfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:04.119650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:04.120514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:04.121274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:04.124559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 15/32] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 12:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:04.125770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53adfb-c7af-4f18-ada0-b6f341464bfd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:04.214857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3c98c-98ae-4228-8619-8c218a1b4d65 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:04.250357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53adfb-c7af-4f18-ada0-b6f341464bfd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:04.255976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2340f-3d6b-4a9b-9cf7-c23b257ea235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:04.256854 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2340f-3d6b-4a9b-9cf7-c23b257ea235 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.257136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2340f-3d6b-4a9b-9cf7-c23b257ea235 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.257408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2340f-3d6b-4a9b-9cf7-c23b257ea235 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.258089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2340f-3d6b-4a9b-9cf7-c23b257ea235 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:04.259515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd2340f-3d6b-4a9b-9cf7-c23b257ea235 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:04.285054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 18/33] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 24 12:59:04 2022] POST /identity/v3/roles => generated 225 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:04.285878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2340f-3d6b-4a9b-9cf7-c23b257ea235 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:04.287267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:04.305180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:04.305823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.306270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.306555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.307257 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:04.307569 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:04.309127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:04.343295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:04.343481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 16/34] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 12:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:04.352919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:04.472613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2340f-3d6b-4a9b-9cf7-c23b257ea235 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:04.499142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf260f8-1ec3-446d-9de9-496f09fe068b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:04.515237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:04.515853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.516163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.516693 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.517188 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:04.517547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:04.518942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:04.530143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 19/35] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:04.535293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:04.547979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fdaf19-5bf6-48ca-9752-f4b015e45201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:04.549511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaf19-5bf6-48ca-9752-f4b015e45201 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.550007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaf19-5bf6-48ca-9752-f4b015e45201 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.550532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fdaf19-5bf6-48ca-9752-f4b015e45201 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.551832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdaf19-5bf6-48ca-9752-f4b015e45201 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:04.553531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fdaf19-5bf6-48ca-9752-f4b015e45201 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:04.593526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 17/36] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 12:59:04 2022] POST /identity/v3/projects => generated 301 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 12:59:04.595653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfd79b-004b-42f7-b026-1529a556c5c8 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.596152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfd79b-004b-42f7-b026-1529a556c5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.596903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfd79b-004b-42f7-b026-1529a556c5c8 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.603799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 18/37] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:04.606797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8807d9-0827-48ef-a71e-0c49218a362d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.607117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8807d9-0827-48ef-a71e-0c49218a362d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.607387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8807d9-0827-48ef-a71e-0c49218a362d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.633096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4a8807d9-0827-48ef-a71e-0c49218a362d None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:04.634882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4a8807d9-0827-48ef-a71e-0c49218a362d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:04.675519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415192a4-e42f-4d99-9306-60fd7747fa7f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:04.680329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:04.689492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:04.732724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 19/38] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:04.737802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9335524-0558-42e5-8cfe-8f7be0650165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:04.738810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335524-0558-42e5-8cfe-8f7be0650165 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.739371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335524-0558-42e5-8cfe-8f7be0650165 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.739862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9335524-0558-42e5-8cfe-8f7be0650165 None admin] PATH_INFO: `/v3/users/c47c715f72104fe1bd8f50179af11065` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.741339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335524-0558-42e5-8cfe-8f7be0650165 None admin] RBAC: Authorizing `identity:get_user(user_id=c47c715f72104fe1bd8f50179af11065)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:04.743043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9335524-0558-42e5-8cfe-8f7be0650165 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:04.745055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 20/39] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:04 2022] GET /identity/v3/users/c47c715f72104fe1bd8f50179af11065 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:04.751355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8807d9-0827-48ef-a71e-0c49218a362d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:04.758072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9d7e7-ba68-4e96-b66c-a9db797822d3 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.758671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9d7e7-ba68-4e96-b66c-a9db797822d3 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.759205 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9d7e7-ba68-4e96-b66c-a9db797822d3 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.760941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 21/40] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:04.769862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bd256-597d-40aa-8b88-ca5c304cff78 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.769862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bd256-597d-40aa-8b88-ca5c304cff78 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.770127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01bd256-597d-40aa-8b88-ca5c304cff78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.802622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c01bd256-597d-40aa-8b88-ca5c304cff78 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:04.803927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c01bd256-597d-40aa-8b88-ca5c304cff78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:04.885131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8807d9-0827-48ef-a71e-0c49218a362d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:04.927925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc07f72-95c2-4a42-b262-2eae9db42215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:04.929412 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc07f72-95c2-4a42-b262-2eae9db42215 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:04.929898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc07f72-95c2-4a42-b262-2eae9db42215 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:04.930402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc07f72-95c2-4a42-b262-2eae9db42215 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:04.931848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc07f72-95c2-4a42-b262-2eae9db42215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:04.933557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc07f72-95c2-4a42-b262-2eae9db42215 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:04.935894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 20/41] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:04.941948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc07f72-95c2-4a42-b262-2eae9db42215 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:04.988350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:04.998398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 22/42] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:05.013874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bd256-597d-40aa-8b88-ca5c304cff78 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:05.057573 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc07f72-95c2-4a42-b262-2eae9db42215 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:05.101420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:05.102030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:05.102312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:05.102617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:05.103380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.domain.id=default, user.id=c47c715f72104fe1bd8f50179af11065) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:05.103690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:05.105867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:05.115835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 21/43] 10.222.0.42 () {56 vars in 1322 bytes} [Mon Oct 24 12:59:04 2022] GET /identity/v3/role_assignments?user.id=c47c715f72104fe1bd8f50179af11065&scope.domain.id=default&role.id=11b74890c7484c6088dd78c106de8be4 => generated 530 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:05.125159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01bd256-597d-40aa-8b88-ca5c304cff78 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:05.127214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:05.167551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e573ed3-fe2b-4955-ba23-0b17ffd57fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:05.168203 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e573ed3-fe2b-4955-ba23-0b17ffd57fa7 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:05.168485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e573ed3-fe2b-4955-ba23-0b17ffd57fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:05.168763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e573ed3-fe2b-4955-ba23-0b17ffd57fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:05.170158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e573ed3-fe2b-4955-ba23-0b17ffd57fa7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:05.171617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e573ed3-fe2b-4955-ba23-0b17ffd57fa7 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:05.188784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 23/44] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 24 12:59:05 2022] POST /identity/v3/roles => generated 223 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 12:59:05.271087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:05.387017 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10306f8-ccc1-461b-b653-b85b7fd61dc5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:05.390092 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:05.422937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ce2ac-42f7-4bf0-969e-00827cccbbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:05.423623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ce2ac-42f7-4bf0-969e-00827cccbbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:05.423915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ce2ac-42f7-4bf0-969e-00827cccbbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:05.424201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ce2ac-42f7-4bf0-969e-00827cccbbb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:05.425329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ce2ac-42f7-4bf0-969e-00827cccbbb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:05.426793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ce2ac-42f7-4bf0-969e-00827cccbbb1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:05.428404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 22/45] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:05.434199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e573ed3-fe2b-4955-ba23-0b17ffd57fa7 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:05.534245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e573ed3-fe2b-4955-ba23-0b17ffd57fa7 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:05.567457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2238025-daa3-4027-91ac-123f6c7f7fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:05.568063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238025-daa3-4027-91ac-123f6c7f7fea None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:05.568352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238025-daa3-4027-91ac-123f6c7f7fea None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:05.568635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2238025-daa3-4027-91ac-123f6c7f7fea None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:05.569292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238025-daa3-4027-91ac-123f6c7f7fea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:05.570734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2238025-daa3-4027-91ac-123f6c7f7fea None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:05.598736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 12:59:05 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:07.719121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eec95-7dfb-44d2-b4ef-6859edc06852 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:07.719637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eec95-7dfb-44d2-b4ef-6859edc06852 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:07.719721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3eec95-7dfb-44d2-b4ef-6859edc06852 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:07.721057 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:07.726283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f8adf-4914-42d5-9641-1555f5e63ce8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:07.726697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f8adf-4914-42d5-9641-1555f5e63ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:07.726836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f8adf-4914-42d5-9641-1555f5e63ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:07.745163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0a7f8adf-4914-42d5-9641-1555f5e63ce8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:07.746333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0a7f8adf-4914-42d5-9641-1555f5e63ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:07.755572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:07.812311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:08.079260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eec95-7dfb-44d2-b4ef-6859edc06852 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:08.178062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3eec95-7dfb-44d2-b4ef-6859edc06852 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:08.211329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee9bf7-d5e4-4e1d-8ca8-d8f21f1db204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:08.211943 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9bf7-d5e4-4e1d-8ca8-d8f21f1db204 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:08.212232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9bf7-d5e4-4e1d-8ca8-d8f21f1db204 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:08.212511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9bf7-d5e4-4e1d-8ca8-d8f21f1db204 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:08.213629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9bf7-d5e4-4e1d-8ca8-d8f21f1db204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:08.215095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9bf7-d5e4-4e1d-8ca8-d8f21f1db204 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:08.216663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:08.222068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f8adf-4914-42d5-9641-1555f5e63ce8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:08.318433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f8adf-4914-42d5-9641-1555f5e63ce8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:08.351432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68679c9e-38fe-4e27-bec1-ec4612ed207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:08.351985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679c9e-38fe-4e27-bec1-ec4612ed207d None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:08.352272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679c9e-38fe-4e27-bec1-ec4612ed207d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:08.352550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68679c9e-38fe-4e27-bec1-ec4612ed207d None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:08.353199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68679c9e-38fe-4e27-bec1-ec4612ed207d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:08.354645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68679c9e-38fe-4e27-bec1-ec4612ed207d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:08.369503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:08.381935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 26/50] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Oct 24 12:59:08 2022] POST /identity/v3/projects => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:10.508233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c6a7d-261b-4772-b22a-50ee4053c283 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:10.508769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c6a7d-261b-4772-b22a-50ee4053c283 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:10.508874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c6a7d-261b-4772-b22a-50ee4053c283 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:10.510142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 25/51] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:10.515388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafb345-9358-4590-9553-6d838cd405de None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:10.515782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafb345-9358-4590-9553-6d838cd405de None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:10.515955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafb345-9358-4590-9553-6d838cd405de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:10.534410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1cafb345-9358-4590-9553-6d838cd405de None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:10.535632 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1cafb345-9358-4590-9553-6d838cd405de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:10.601696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 27/52] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:10.874359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c6a7d-261b-4772-b22a-50ee4053c283 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:10.918002 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:11.085335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c6a7d-261b-4772-b22a-50ee4053c283 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:11.118616 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a82f0-fc25-41aa-9020-751fa62355b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:11.119268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a82f0-fc25-41aa-9020-751fa62355b3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:11.119559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a82f0-fc25-41aa-9020-751fa62355b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:11.119841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a82f0-fc25-41aa-9020-751fa62355b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:11.120952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a82f0-fc25-41aa-9020-751fa62355b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:11.123031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a82f0-fc25-41aa-9020-751fa62355b3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:11.124676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 26/53] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:11.130211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafb345-9358-4590-9553-6d838cd405de None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:11.226578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafb345-9358-4590-9553-6d838cd405de None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:11.251830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:11.260988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513929f-a4c9-44db-bb3a-920232ff12ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:11.261621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513929f-a4c9-44db-bb3a-920232ff12ba None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:11.261905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513929f-a4c9-44db-bb3a-920232ff12ba None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:11.262179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513929f-a4c9-44db-bb3a-920232ff12ba None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:11.262842 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513929f-a4c9-44db-bb3a-920232ff12ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:11.264331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513929f-a4c9-44db-bb3a-920232ff12ba None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:11.290936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 28/54] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:11 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:14.482663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbadc8a-4a2c-44d2-99ed-ad9af2c925bb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:14.483199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbadc8a-4a2c-44d2-99ed-ad9af2c925bb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:14.483271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbadc8a-4a2c-44d2-99ed-ad9af2c925bb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:14.484949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 27/55] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:14.490957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04be1-f07c-4b7a-8841-2ef6db5719d6 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:14.491514 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04be1-f07c-4b7a-8841-2ef6db5719d6 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:14.491596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04be1-f07c-4b7a-8841-2ef6db5719d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:14.516179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-01c04be1-f07c-4b7a-8841-2ef6db5719d6 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:14.517418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-01c04be1-f07c-4b7a-8841-2ef6db5719d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:14.608069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086cca-c2f0-4351-96ba-16e99a7e76d4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:14.608723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086cca-c2f0-4351-96ba-16e99a7e76d4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:14.609202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa086cca-c2f0-4351-96ba-16e99a7e76d4 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:14.610728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 28/56] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:59:14.615663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eafbb-962f-4814-b51e-a00278dc1d4c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:14.616162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eafbb-962f-4814-b51e-a00278dc1d4c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:14.616731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eafbb-962f-4814-b51e-a00278dc1d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:14.617944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 29/57] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:14.648310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bf0eafbb-962f-4814-b51e-a00278dc1d4c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:14.649837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bf0eafbb-962f-4814-b51e-a00278dc1d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:14.751806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 29/58] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:14.863374 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836483ae-e14d-40d6-ae18-a8f0a7a21ff1 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:14.863705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836483ae-e14d-40d6-ae18-a8f0a7a21ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:14.863938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836483ae-e14d-40d6-ae18-a8f0a7a21ff1 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:14.865292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 30/59] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:14.872188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3bf53-6572-4e8d-abdc-3e2633e5b536 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:14.872774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3bf53-6572-4e8d-abdc-3e2633e5b536 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:14.873322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3bf53-6572-4e8d-abdc-3e2633e5b536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:14.908080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a6e3bf53-6572-4e8d-abdc-3e2633e5b536 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:14.909357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a6e3bf53-6572-4e8d-abdc-3e2633e5b536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:14.939900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:14.988049 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 30/60] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:15.010380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836483ae-e14d-40d6-ae18-a8f0a7a21ff1 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.175434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836483ae-e14d-40d6-ae18-a8f0a7a21ff1 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:15.179733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d468c-9f81-4fed-9d9a-50e73a6afcc3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.180290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d468c-9f81-4fed-9d9a-50e73a6afcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.180783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d468c-9f81-4fed-9d9a-50e73a6afcc3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.182417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 31/61] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:15.191903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:15.195491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6698f-4e86-4a0c-ae6b-8509aaadf138 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.195821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6698f-4e86-4a0c-ae6b-8509aaadf138 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.198827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc6698f-4e86-4a0c-ae6b-8509aaadf138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.216992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5dc6698f-4e86-4a0c-ae6b-8509aaadf138 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:15.218488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5dc6698f-4e86-4a0c-ae6b-8509aaadf138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:15.225185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a153a2-1032-481e-bffd-3b6337891abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:15.225829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a153a2-1032-481e-bffd-3b6337891abb None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.226112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a153a2-1032-481e-bffd-3b6337891abb None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.226385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a153a2-1032-481e-bffd-3b6337891abb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.239882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a153a2-1032-481e-bffd-3b6337891abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-f4a153a2-1032-481e-bffd-3b6337891abb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:15.246559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:15.247647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:15.248588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 31/62] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:15.249389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a153a2-1032-481e-bffd-3b6337891abb None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.299103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 32/63] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:15.300154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6698f-4e86-4a0c-ae6b-8509aaadf138 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.368431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a153a2-1032-481e-bffd-3b6337891abb None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:15.406554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19707f7b-44d3-46f4-8f37-496acaee41b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:15.407549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19707f7b-44d3-46f4-8f37-496acaee41b6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.407976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19707f7b-44d3-46f4-8f37-496acaee41b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.408391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19707f7b-44d3-46f4-8f37-496acaee41b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.409609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:15.413877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc6698f-4e86-4a0c-ae6b-8509aaadf138 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:15.421501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19707f7b-44d3-46f4-8f37-496acaee41b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-19707f7b-44d3-46f4-8f37-496acaee41b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:15.427210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:15.428098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:15.428866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:15.431285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 32/64] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:15.432235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19707f7b-44d3-46f4-8f37-496acaee41b6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.460698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:15.461319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.461631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.461901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.463012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:15.463357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:15.464763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:15.471868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:15.477858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 33/65] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:15.479649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.551369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19707f7b-44d3-46f4-8f37-496acaee41b6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:15.585249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f87379-1884-473e-819f-5a2b1fedcd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:15.585991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87379-1884-473e-819f-5a2b1fedcd07 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.586199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87379-1884-473e-819f-5a2b1fedcd07 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.586479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f87379-1884-473e-819f-5a2b1fedcd07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.587966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a860a-8540-413c-bc29-30cf412d8239 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:15.597449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f87379-1884-473e-819f-5a2b1fedcd07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-86f87379-1884-473e-819f-5a2b1fedcd07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:15.602347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:15.603250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:15.604019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 33/66] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:15.604646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc1112-8f88-44c6-a85b-f0d34ac5438c None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.605196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc1112-8f88-44c6-a85b-f0d34ac5438c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.605196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc1112-8f88-44c6-a85b-f0d34ac5438c None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.611197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 34/67] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:15.611197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc1112-8f88-44c6-a85b-f0d34ac5438c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.624760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:15.625374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.625660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.625943 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.626616 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:15.627079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:15.635145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:15.643332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 34/68] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:15.644129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.669232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:15.704847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:15.714796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc1112-8f88-44c6-a85b-f0d34ac5438c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:15.747737 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcf7e3-9014-4786-9f2b-80ff2ec554da None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:15.748759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ab200-8213-4d39-bd90-f062378ed821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:15.749391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ab200-8213-4d39-bd90-f062378ed821 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.749679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ab200-8213-4d39-bd90-f062378ed821 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.749955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149ab200-8213-4d39-bd90-f062378ed821 None admin] PATH_INFO: `/v3/users/826adfd9b22449ba95ba52818bac30d3` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.764006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ab200-8213-4d39-bd90-f062378ed821 None admin] RBAC: Authorizing `identity:get_user(user_id=826adfd9b22449ba95ba52818bac30d3)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:15.765480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149ab200-8213-4d39-bd90-f062378ed821 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:15.767120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 35/69] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/users/826adfd9b22449ba95ba52818bac30d3 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:15.767898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ab200-8213-4d39-bd90-f062378ed821 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.784177 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba06845-fc5f-4002-aeae-80e774395f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:15.784821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba06845-fc5f-4002-aeae-80e774395f63 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.785110 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba06845-fc5f-4002-aeae-80e774395f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.785421 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba06845-fc5f-4002-aeae-80e774395f63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.795832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba06845-fc5f-4002-aeae-80e774395f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-bba06845-fc5f-4002-aeae-80e774395f63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:15.801106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:15.802142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:15.803861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 35/70] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:15.804520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5339-268b-47f6-a0ec-ba03069843f7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.804520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5339-268b-47f6-a0ec-ba03069843f7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.804520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5339-268b-47f6-a0ec-ba03069843f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.821669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-201d5339-268b-47f6-a0ec-ba03069843f7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:15.822866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-201d5339-268b-47f6-a0ec-ba03069843f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:15.872617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149ab200-8213-4d39-bd90-f062378ed821 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:15.889913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 36/71] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:15.890744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5339-268b-47f6-a0ec-ba03069843f7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.904876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:15.922513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:15.924118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:15.924775 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:15.925263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:15.926634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:15.928293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:15.930846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:15.942134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 36/72] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:15.942936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:15.958249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:16.009105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5339-268b-47f6-a0ec-ba03069843f7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.049270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427571c3-2dd7-4eab-9251-a4c88fd49126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.049919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427571c3-2dd7-4eab-9251-a4c88fd49126 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.050212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427571c3-2dd7-4eab-9251-a4c88fd49126 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.050497 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427571c3-2dd7-4eab-9251-a4c88fd49126 None admin] PATH_INFO: `/v3/users/c47c715f72104fe1bd8f50179af11065` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.050696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d5239-301a-4520-a165-05c6960270b1 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.051917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427571c3-2dd7-4eab-9251-a4c88fd49126 None admin] RBAC: Authorizing `identity:get_user(user_id=c47c715f72104fe1bd8f50179af11065)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.053446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427571c3-2dd7-4eab-9251-a4c88fd49126 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.054995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/users/c47c715f72104fe1bd8f50179af11065 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:16.055798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427571c3-2dd7-4eab-9251-a4c88fd49126 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.088163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b81221-7924-42e3-be78-a49b22ab5264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.088817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81221-7924-42e3-be78-a49b22ab5264 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.089092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81221-7924-42e3-be78-a49b22ab5264 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.089381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b81221-7924-42e3-be78-a49b22ab5264 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.090692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b81221-7924-42e3-be78-a49b22ab5264 None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.092197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b81221-7924-42e3-be78-a49b22ab5264 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.093883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 37/74] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:15 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:16.094651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b81221-7924-42e3-be78-a49b22ab5264 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.151531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:16.172545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427571c3-2dd7-4eab-9251-a4c88fd49126 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.187961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:16.206683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.207330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.207620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.207900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.207992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b81221-7924-42e3-be78-a49b22ab5264 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.208575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:16.208884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.210281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.219193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 38/75] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:16.219991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.242796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e637bd-cd87-4726-bec6-1198ca5ce562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.243449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e637bd-cd87-4726-bec6-1198ca5ce562 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.243730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e637bd-cd87-4726-bec6-1198ca5ce562 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.244005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e637bd-cd87-4726-bec6-1198ca5ce562 None admin] PATH_INFO: `/v3/users/826adfd9b22449ba95ba52818bac30d3` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.245140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e637bd-cd87-4726-bec6-1198ca5ce562 None admin] RBAC: Authorizing `identity:get_user(user_id=826adfd9b22449ba95ba52818bac30d3)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.247299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e637bd-cd87-4726-bec6-1198ca5ce562 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.249047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 38/76] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:16 2022] GET /identity/v3/users/826adfd9b22449ba95ba52818bac30d3 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:16.249781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e637bd-cd87-4726-bec6-1198ca5ce562 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.326262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c43d45-0d64-44b2-8377-5db2ab89b8f0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.351617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e637bd-cd87-4726-bec6-1198ca5ce562 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.359473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4777e4d-70eb-478e-ae3f-a20df6398d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.360102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4777e4d-70eb-478e-ae3f-a20df6398d92 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.360396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4777e4d-70eb-478e-ae3f-a20df6398d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.360676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4777e4d-70eb-478e-ae3f-a20df6398d92 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.361830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4777e4d-70eb-478e-ae3f-a20df6398d92 None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.363312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4777e4d-70eb-478e-ae3f-a20df6398d92 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.364946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 39/77] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:16 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:16.369799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4777e4d-70eb-478e-ae3f-a20df6398d92 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.382050 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:16.385585 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791c34c-372c-4638-b3e9-eff85edff40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.386259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791c34c-372c-4638-b3e9-eff85edff40b None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.386548 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791c34c-372c-4638-b3e9-eff85edff40b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.386829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791c34c-372c-4638-b3e9-eff85edff40b None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2/users/826adfd9b22449ba95ba52818bac30d3/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.389373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791c34c-372c-4638-b3e9-eff85edff40b None admin] RBAC: Authorizing `identity:create_grant(project_id=1634123c90c5493296570578db87c5c2, user_id=826adfd9b22449ba95ba52818bac30d3, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.390828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791c34c-372c-4638-b3e9-eff85edff40b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.403107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 39/78] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:16 2022] PUT /identity/v3/projects/1634123c90c5493296570578db87c5c2/users/826adfd9b22449ba95ba52818bac30d3/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:16.403899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791c34c-372c-4638-b3e9-eff85edff40b None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.417219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:16.490512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4777e4d-70eb-478e-ae3f-a20df6398d92 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.528127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf28be-c6d7-4e46-9204-bf8cfe4965c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.528726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791c34c-372c-4638-b3e9-eff85edff40b None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.528844 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf28be-c6d7-4e46-9204-bf8cfe4965c2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.529062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf28be-c6d7-4e46-9204-bf8cfe4965c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.529342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf28be-c6d7-4e46-9204-bf8cfe4965c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.530840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf28be-c6d7-4e46-9204-bf8cfe4965c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.533452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf28be-c6d7-4e46-9204-bf8cfe4965c2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.535339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 40/79] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:16.536051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf28be-c6d7-4e46-9204-bf8cfe4965c2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.569880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9d5e0-8687-4d22-bfa0-5b511c4f9b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.570960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9d5e0-8687-4d22-bfa0-5b511c4f9b3e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.571975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9d5e0-8687-4d22-bfa0-5b511c4f9b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.572260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9d5e0-8687-4d22-bfa0-5b511c4f9b3e None admin] PATH_INFO: `/v3/users/826adfd9b22449ba95ba52818bac30d3` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.573889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9d5e0-8687-4d22-bfa0-5b511c4f9b3e None admin] RBAC: Authorizing `identity:get_user(user_id=826adfd9b22449ba95ba52818bac30d3)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.576124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd9d5e0-8687-4d22-bfa0-5b511c4f9b3e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.577952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 40/80] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:16 2022] GET /identity/v3/users/826adfd9b22449ba95ba52818bac30d3 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:16.578614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9d5e0-8687-4d22-bfa0-5b511c4f9b3e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.627673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:16.650353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf28be-c6d7-4e46-9204-bf8cfe4965c2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.665550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:16.686971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75406906-fb2d-47a1-ab3f-bb3e95c6f957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.687646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406906-fb2d-47a1-ab3f-bb3e95c6f957 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.687953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406906-fb2d-47a1-ab3f-bb3e95c6f957 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.688270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75406906-fb2d-47a1-ab3f-bb3e95c6f957 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.689433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75406906-fb2d-47a1-ab3f-bb3e95c6f957 None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.690918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75406906-fb2d-47a1-ab3f-bb3e95c6f957 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.697507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 41/81] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:16 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:16.697654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9d5e0-8687-4d22-bfa0-5b511c4f9b3e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.697735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75406906-fb2d-47a1-ab3f-bb3e95c6f957 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.733434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49662f53-78f9-4b9c-ba3a-593e8cc4f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.734115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49662f53-78f9-4b9c-ba3a-593e8cc4f514 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.734403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49662f53-78f9-4b9c-ba3a-593e8cc4f514 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.734689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49662f53-78f9-4b9c-ba3a-593e8cc4f514 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2/users/c47c715f72104fe1bd8f50179af11065/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.736645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49662f53-78f9-4b9c-ba3a-593e8cc4f514 None admin] RBAC: Authorizing `identity:create_grant(project_id=1634123c90c5493296570578db87c5c2, user_id=c47c715f72104fe1bd8f50179af11065, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.738093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49662f53-78f9-4b9c-ba3a-593e8cc4f514 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.766586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 41/82] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:16 2022] PUT /identity/v3/projects/1634123c90c5493296570578db87c5c2/users/c47c715f72104fe1bd8f50179af11065/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:16.766586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49662f53-78f9-4b9c-ba3a-593e8cc4f514 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.825795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75406906-fb2d-47a1-ab3f-bb3e95c6f957 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.866564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7caf3-a7f7-4223-ba2e-97dec1244e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.867336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7caf3-a7f7-4223-ba2e-97dec1244e67 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.867721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7caf3-a7f7-4223-ba2e-97dec1244e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.868105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7caf3-a7f7-4223-ba2e-97dec1244e67 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:16.868879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7caf3-a7f7-4223-ba2e-97dec1244e67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:16.870519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a7caf3-a7f7-4223-ba2e-97dec1244e67 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:16.911121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 42/83] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:16 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:16.911121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7caf3-a7f7-4223-ba2e-97dec1244e67 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:16.911673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49662f53-78f9-4b9c-ba3a-593e8cc4f514 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:16.917598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:16.997224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1cce1-e9d6-4ebb-94a3-dd59f71bdc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:16.998450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1cce1-e9d6-4ebb-94a3-dd59f71bdc19 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:16.998740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1cce1-e9d6-4ebb-94a3-dd59f71bdc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:16.999021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1cce1-e9d6-4ebb-94a3-dd59f71bdc19 None admin] PATH_INFO: `/v3/projects/4f46937fc7374b7fb2f1dbd0b4ea65cb` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:17.000400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1cce1-e9d6-4ebb-94a3-dd59f71bdc19 None admin] RBAC: Authorizing `identity:get_project(project_id=4f46937fc7374b7fb2f1dbd0b4ea65cb)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:17.001897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1cce1-e9d6-4ebb-94a3-dd59f71bdc19 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:17.004506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 42/84] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:16 2022] GET /identity/v3/projects/4f46937fc7374b7fb2f1dbd0b4ea65cb => generated 312 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:17.013133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1cce1-e9d6-4ebb-94a3-dd59f71bdc19 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:17.018966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:17.105877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7caf3-a7f7-4223-ba2e-97dec1244e67 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:17.157703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a484d22-30b3-4a8e-83db-900b1a0642cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:17.158489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484d22-30b3-4a8e-83db-900b1a0642cc None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:17.158847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484d22-30b3-4a8e-83db-900b1a0642cc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:17.159220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a484d22-30b3-4a8e-83db-900b1a0642cc None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2/users/826adfd9b22449ba95ba52818bac30d3/roles/78ff1dcdc39743048089711fcfc4d3a4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:17.162925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a484d22-30b3-4a8e-83db-900b1a0642cc None admin] RBAC: Authorizing `identity:create_grant(project_id=1634123c90c5493296570578db87c5c2, user_id=826adfd9b22449ba95ba52818bac30d3, role_id=78ff1dcdc39743048089711fcfc4d3a4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:17.164498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a484d22-30b3-4a8e-83db-900b1a0642cc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:17.180062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1cce1-e9d6-4ebb-94a3-dd59f71bdc19 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:17.194808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 43/85] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:16 2022] PUT /identity/v3/projects/1634123c90c5493296570578db87c5c2/users/826adfd9b22449ba95ba52818bac30d3/roles/78ff1dcdc39743048089711fcfc4d3a4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:17.226327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36da058-a46e-4030-b64a-abea0c212637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:17.227021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36da058-a46e-4030-b64a-abea0c212637 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:17.227341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36da058-a46e-4030-b64a-abea0c212637 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:17.227624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36da058-a46e-4030-b64a-abea0c212637 None admin] PATH_INFO: `/v3/projects/4f46937fc7374b7fb2f1dbd0b4ea65cb/users/826adfd9b22449ba95ba52818bac30d3/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:17.229655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36da058-a46e-4030-b64a-abea0c212637 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f46937fc7374b7fb2f1dbd0b4ea65cb, user_id=826adfd9b22449ba95ba52818bac30d3, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:17.231751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36da058-a46e-4030-b64a-abea0c212637 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:17.250945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 43/86] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:17 2022] PUT /identity/v3/projects/4f46937fc7374b7fb2f1dbd0b4ea65cb/users/826adfd9b22449ba95ba52818bac30d3/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 12:59:19.149066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0338364-7e7c-4f56-9a47-8009ded3aa3d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:19.150076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0338364-7e7c-4f56-9a47-8009ded3aa3d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:19.150589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0338364-7e7c-4f56-9a47-8009ded3aa3d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:19.152182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 44/87] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:19.157516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e652c8-f12a-4d9c-8ead-d35f6688e843 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:19.157916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e652c8-f12a-4d9c-8ead-d35f6688e843 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:19.158082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e652c8-f12a-4d9c-8ead-d35f6688e843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:19.177095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c3e652c8-f12a-4d9c-8ead-d35f6688e843 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:19.178289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c3e652c8-f12a-4d9c-8ead-d35f6688e843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:19.254891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:19.270012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 44/88] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:19.536399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f15677-f2ef-4666-99f8-59b31f26c595 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:19.537080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f15677-f2ef-4666-99f8-59b31f26c595 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:19.537601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f15677-f2ef-4666-99f8-59b31f26c595 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:19.539416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 45/89] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:19.551913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d280a-f00c-45cf-83d4-fd2d0a160c59 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:19.552204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d280a-f00c-45cf-83d4-fd2d0a160c59 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:19.552480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d280a-f00c-45cf-83d4-fd2d0a160c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:19.576069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-960d280a-f00c-45cf-83d4-fd2d0a160c59 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:19.577264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-960d280a-f00c-45cf-83d4-fd2d0a160c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:19.675010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 45/90] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:19.710896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f15677-f2ef-4666-99f8-59b31f26c595 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:19.823509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:19.868083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f15677-f2ef-4666-99f8-59b31f26c595 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:19.914610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062a552-887b-4e46-9b2f-bd19822debee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:19.915570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062a552-887b-4e46-9b2f-bd19822debee None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:19.916045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062a552-887b-4e46-9b2f-bd19822debee None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:19.916501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062a552-887b-4e46-9b2f-bd19822debee None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:19.930885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062a552-887b-4e46-9b2f-bd19822debee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a062a552-887b-4e46-9b2f-bd19822debee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:19.936641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:19.937491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:19.938261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 46/91] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:19.943742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d280a-f00c-45cf-83d4-fd2d0a160c59 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:20.080309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d280a-f00c-45cf-83d4-fd2d0a160c59 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:20.104321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:20.119334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:20.120104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.120608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.120888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.122149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] RBAC: Adding query filter params (name=member) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:20.122465 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:20.128559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:20.138752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 46/92] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:20.144442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062a552-887b-4e46-9b2f-bd19822debee None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:20.197555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:20.327693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062a552-887b-4e46-9b2f-bd19822debee None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:20.355671 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:20.377827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5ba92-c7fd-4f81-8f6f-27be11a101cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:20.378476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ba92-c7fd-4f81-8f6f-27be11a101cf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.378762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ba92-c7fd-4f81-8f6f-27be11a101cf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.379036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ba92-c7fd-4f81-8f6f-27be11a101cf None admin] PATH_INFO: `/v3/users/826adfd9b22449ba95ba52818bac30d3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.381783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ba92-c7fd-4f81-8f6f-27be11a101cf None admin] RBAC: Authorizing `identity:get_user(user_id=826adfd9b22449ba95ba52818bac30d3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:20.383821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ba92-c7fd-4f81-8f6f-27be11a101cf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:20.384742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bb8f4-cf67-4870-a5df-554ad867e242 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:20.385722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 47/93] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:20 2022] GET /identity/v3/users/826adfd9b22449ba95ba52818bac30d3 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:20.386901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae001-6d58-4d87-8894-b81095fc8ffa None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.387218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae001-6d58-4d87-8894-b81095fc8ffa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.387524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ae001-6d58-4d87-8894-b81095fc8ffa None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.388764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 48/94] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:20.400042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae001-6d58-4d87-8894-b81095fc8ffa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:20.450168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230540d-9b0a-456f-a161-0af48e498311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:20.451093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230540d-9b0a-456f-a161-0af48e498311 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.451378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230540d-9b0a-456f-a161-0af48e498311 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.451663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230540d-9b0a-456f-a161-0af48e498311 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.464806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230540d-9b0a-456f-a161-0af48e498311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-d230540d-9b0a-456f-a161-0af48e498311 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:20.473215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:20.474068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:20.474825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 47/95] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 12:59:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:20.476226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76f9b-e224-4e43-8072-6276f8009dd4 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.476226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76f9b-e224-4e43-8072-6276f8009dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.476226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e76f9b-e224-4e43-8072-6276f8009dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.498074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-81e76f9b-e224-4e43-8072-6276f8009dd4 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:20.499627 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-81e76f9b-e224-4e43-8072-6276f8009dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:20.540085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75ae001-6d58-4d87-8894-b81095fc8ffa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:20.560422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:20.581602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f00ce-ec7c-4372-bf89-cae56cc0f369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:20.582303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f00ce-ec7c-4372-bf89-cae56cc0f369 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.582692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f00ce-ec7c-4372-bf89-cae56cc0f369 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.582977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f00ce-ec7c-4372-bf89-cae56cc0f369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.594952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 48/96] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:20.597355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb5855-bf5c-4d73-8d56-bc36878c520a None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.597916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb5855-bf5c-4d73-8d56-bc36878c520a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.598476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb5855-bf5c-4d73-8d56-bc36878c520a None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.600233 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 49/97] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:20.605448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb5855-bf5c-4d73-8d56-bc36878c520a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:20.706906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:20.723225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f00ce-ec7c-4372-bf89-cae56cc0f369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:20.726890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f00ce-ec7c-4372-bf89-cae56cc0f369 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:20.763086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb5855-bf5c-4d73-8d56-bc36878c520a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:20.800189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025c3c6-75e5-46e3-9974-14a5bd16da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:20.800832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025c3c6-75e5-46e3-9974-14a5bd16da17 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.801186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025c3c6-75e5-46e3-9974-14a5bd16da17 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.801468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025c3c6-75e5-46e3-9974-14a5bd16da17 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.802139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025c3c6-75e5-46e3-9974-14a5bd16da17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:20.802779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025c3c6-75e5-46e3-9974-14a5bd16da17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:20.803952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025c3c6-75e5-46e3-9974-14a5bd16da17 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:20.814435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 50/98] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:20.819250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db27285-e14e-4ca9-bde8-78777df2788b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.819250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db27285-e14e-4ca9-bde8-78777df2788b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.819250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db27285-e14e-4ca9-bde8-78777df2788b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.844236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5db27285-e14e-4ca9-bde8-78777df2788b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:20.846621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5db27285-e14e-4ca9-bde8-78777df2788b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:20.855743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 49/99] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:20.861636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb726-f11e-44e5-b334-515298f5779d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:20.861973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb726-f11e-44e5-b334-515298f5779d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:20.862235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbdb726-f11e-44e5-b334-515298f5779d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:20.863862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 50/100] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:20.864402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdb726-f11e-44e5-b334-515298f5779d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:20.887824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:20.927616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 51/101] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:20.928398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db27285-e14e-4ca9-bde8-78777df2788b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:20.963418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:20.978054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbdb726-f11e-44e5-b334-515298f5779d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.016647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8b212-82a2-473b-a456-9db2f1895470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.017285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b212-82a2-473b-a456-9db2f1895470 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.017576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b212-82a2-473b-a456-9db2f1895470 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.017860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8b212-82a2-473b-a456-9db2f1895470 None admin] PATH_INFO: `/v3/users/c47c715f72104fe1bd8f50179af11065` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.019014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8b212-82a2-473b-a456-9db2f1895470 None admin] RBAC: Authorizing `identity:get_user(user_id=c47c715f72104fe1bd8f50179af11065)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:21.020498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8b212-82a2-473b-a456-9db2f1895470 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:21.022110 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 51/102] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:20 2022] GET /identity/v3/users/c47c715f72104fe1bd8f50179af11065 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:21.023276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db1f2-d03c-4fe7-9e21-177383d9ed70 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.023567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db1f2-d03c-4fe7-9e21-177383d9ed70 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.023846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db1f2-d03c-4fe7-9e21-177383d9ed70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.036492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db27285-e14e-4ca9-bde8-78777df2788b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.041361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4d5db1f2-d03c-4fe7-9e21-177383d9ed70 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:21.042564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4d5db1f2-d03c-4fe7-9e21-177383d9ed70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:21.089495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a852fa3-f892-4fab-bb9e-862ff0cb8f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.091129 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a852fa3-f892-4fab-bb9e-862ff0cb8f40 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.091567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a852fa3-f892-4fab-bb9e-862ff0cb8f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.092007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a852fa3-f892-4fab-bb9e-862ff0cb8f40 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.093815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a852fa3-f892-4fab-bb9e-862ff0cb8f40 None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:21.095968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a852fa3-f892-4fab-bb9e-862ff0cb8f40 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:21.098063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 52/103] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:20 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:21.099844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1989a4a-395d-405a-ba27-63673b0d8243 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.100550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1989a4a-395d-405a-ba27-63673b0d8243 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.100969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1989a4a-395d-405a-ba27-63673b0d8243 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.102793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 53/104] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:21.103945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1989a4a-395d-405a-ba27-63673b0d8243 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.124285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 52/105] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:21.125142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db1f2-d03c-4fe7-9e21-177383d9ed70 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.168865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:21.222516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:21.236757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db1f2-d03c-4fe7-9e21-177383d9ed70 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.237462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1989a4a-395d-405a-ba27-63673b0d8243 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.272994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b483652-e280-4c4b-8ca8-b0a626f1850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.273598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b483652-e280-4c4b-8ca8-b0a626f1850c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.273699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e083f-304c-4844-baa9-360fd04e70ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.273887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b483652-e280-4c4b-8ca8-b0a626f1850c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.274222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b483652-e280-4c4b-8ca8-b0a626f1850c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.274313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e083f-304c-4844-baa9-360fd04e70ed None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.274524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e083f-304c-4844-baa9-360fd04e70ed None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.274806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60e083f-304c-4844-baa9-360fd04e70ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.275941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e083f-304c-4844-baa9-360fd04e70ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:21.277500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60e083f-304c-4844-baa9-360fd04e70ed None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:21.279264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 54/106] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:21.279878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e083f-304c-4844-baa9-360fd04e70ed None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.351108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b483652-e280-4c4b-8ca8-b0a626f1850c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:21.353185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b483652-e280-4c4b-8ca8-b0a626f1850c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:21.385915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60e083f-304c-4844-baa9-360fd04e70ed None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.388417 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:21.423930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f2185-9bf3-4ad4-8142-7b90eda49ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.424546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f2185-9bf3-4ad4-8142-7b90eda49ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.424832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f2185-9bf3-4ad4-8142-7b90eda49ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.425112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f2185-9bf3-4ad4-8142-7b90eda49ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.425849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f2185-9bf3-4ad4-8142-7b90eda49ea7 None admin] RBAC: Adding query filter params (role.id=54e93caca0fb4c59a01ae64a0ed3a610, scope.project.id=1634123c90c5493296570578db87c5c2, user.id=826adfd9b22449ba95ba52818bac30d3) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:21.426164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f2185-9bf3-4ad4-8142-7b90eda49ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:21.427644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f2185-9bf3-4ad4-8142-7b90eda49ea7 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:21.435468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 55/107] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:21 2022] GET /identity/v3/role_assignments?user.id=826adfd9b22449ba95ba52818bac30d3&scope.project.id=1634123c90c5493296570578db87c5c2&role.id=54e93caca0fb4c59a01ae64a0ed3a610 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:21.436602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4a3c-362a-47be-84e5-06226933c96d None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.436894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4a3c-362a-47be-84e5-06226933c96d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.437175 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3c4a3c-362a-47be-84e5-06226933c96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.454682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:21.464711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1b3c4a3c-362a-47be-84e5-06226933c96d None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:21.466272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1b3c4a3c-362a-47be-84e5-06226933c96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:21.470830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 53/108] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:21.475266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b483652-e280-4c4b-8ca8-b0a626f1850c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.546882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 56/109] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:21.547784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c4a3c-362a-47be-84e5-06226933c96d None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.605920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b483652-e280-4c4b-8ca8-b0a626f1850c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.657407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d552bd-9300-449f-9634-90d2e79797ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.658078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d552bd-9300-449f-9634-90d2e79797ff None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.658369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d552bd-9300-449f-9634-90d2e79797ff None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.658652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d552bd-9300-449f-9634-90d2e79797ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.672208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d552bd-9300-449f-9634-90d2e79797ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-53d552bd-9300-449f-9634-90d2e79797ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:21.678534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:21.679428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:21.685417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:21.686143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:21.686226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 54/110] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:21.686226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d552bd-9300-449f-9634-90d2e79797ff None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.688607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3c4a3c-362a-47be-84e5-06226933c96d None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.727398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4dda0-4790-4dfe-98ad-d22872415d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.728012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4dda0-4790-4dfe-98ad-d22872415d72 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.729291 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4dda0-4790-4dfe-98ad-d22872415d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.729577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4dda0-4790-4dfe-98ad-d22872415d72 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.730231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4dda0-4790-4dfe-98ad-d22872415d72 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:21.731783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb4dda0-4790-4dfe-98ad-d22872415d72 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:21.745211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:21.758485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 57/111] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 12:59:21 2022] POST /identity/v3/projects => generated 302 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:21.759337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4dda0-4790-4dfe-98ad-d22872415d72 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.819463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d552bd-9300-449f-9634-90d2e79797ff None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.860882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b4207-5e44-43c6-bb3a-881d2596799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.861450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b4207-5e44-43c6-bb3a-881d2596799e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.861735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b4207-5e44-43c6-bb3a-881d2596799e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.862013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b4207-5e44-43c6-bb3a-881d2596799e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.875395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b4207-5e44-43c6-bb3a-881d2596799e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-977b4207-5e44-43c6-bb3a-881d2596799e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:21.881197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:21.882348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:21.883141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 55/112] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 12:59:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:21.888583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b4207-5e44-43c6-bb3a-881d2596799e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.888733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb4dda0-4790-4dfe-98ad-d22872415d72 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:21.925958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d8f91-6c61-43e7-b83a-18ef36241f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:21.926615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8f91-6c61-43e7-b83a-18ef36241f81 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:21.926914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8f91-6c61-43e7-b83a-18ef36241f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:21.927216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d8f91-6c61-43e7-b83a-18ef36241f81 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:21.928460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d8f91-6c61-43e7-b83a-18ef36241f81 None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:21.929982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d8f91-6c61-43e7-b83a-18ef36241f81 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:21.931732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 58/113] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:21 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:21.932539 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d8f91-6c61-43e7-b83a-18ef36241f81 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:21.954653 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:22.014664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b4207-5e44-43c6-bb3a-881d2596799e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.025153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:22.047752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d8f91-6c61-43e7-b83a-18ef36241f81 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.055644 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.056262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.056549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.057191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.057867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:22.058473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.061649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.074085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 56/114] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:22.074085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.091950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168126f-7b0a-476b-a5ce-3a1cccf797c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.092589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168126f-7b0a-476b-a5ce-3a1cccf797c6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.092875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168126f-7b0a-476b-a5ce-3a1cccf797c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.093153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168126f-7b0a-476b-a5ce-3a1cccf797c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.103470 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168126f-7b0a-476b-a5ce-3a1cccf797c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-4168126f-7b0a-476b-a5ce-3a1cccf797c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:22.109321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:22.110220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:22.111018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 59/115] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:22.111695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168126f-7b0a-476b-a5ce-3a1cccf797c6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.190848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abaab55-27d4-4da9-a592-a1d5f6eedb76 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.197433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:22.220107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168126f-7b0a-476b-a5ce-3a1cccf797c6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.225269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.225888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.226171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.226448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.227149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:22.227462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.228869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.237467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 57/116] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:22.238273 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.264349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.265388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.265826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.266238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.267158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=1634123c90c5493296570578db87c5c2, user.id=c47c715f72104fe1bd8f50179af11065) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:22.267485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.269320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.273742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:22.275954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 60/117] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/role_assignments?user.id=c47c715f72104fe1bd8f50179af11065&scope.project.id=1634123c90c5493296570578db87c5c2&role.id=11b74890c7484c6088dd78c106de8be4 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:22.284808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.355071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d4966-7260-48c0-a8c7-02be0f862192 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.400120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d2c94-d4a8-4722-9ac2-724e6f696cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.401790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d2c94-d4a8-4722-9ac2-724e6f696cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.402264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d2c94-d4a8-4722-9ac2-724e6f696cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.403383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d2c94-d4a8-4722-9ac2-724e6f696cf1 None admin] PATH_INFO: `/v3/users/85d813f29e9b43528ce7905c0d7cb0e8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.426679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d2c94-d4a8-4722-9ac2-724e6f696cf1 None admin] RBAC: Authorizing `identity:get_user(user_id=85d813f29e9b43528ce7905c0d7cb0e8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.428382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d2c94-d4a8-4722-9ac2-724e6f696cf1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.429197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca355c-bd21-4658-978a-d4e42125c052 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.435129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 58/118] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/users/85d813f29e9b43528ce7905c0d7cb0e8 => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:22.435129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d2c94-d4a8-4722-9ac2-724e6f696cf1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.485774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.486408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.486696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.488072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.488072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] RBAC: Adding query filter params (name=member) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:22.488371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.489651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.500231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 61/119] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:22.501091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.508058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:22.560100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:22.571497 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d2c94-d4a8-4722-9ac2-724e6f696cf1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.608957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dd934-3d08-4a4b-ac5b-3b92afc10c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.609598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dd934-3d08-4a4b-ac5b-3b92afc10c44 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.609878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dd934-3d08-4a4b-ac5b-3b92afc10c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.610179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0dd934-3d08-4a4b-ac5b-3b92afc10c44 None admin] PATH_INFO: `/v3/users/826adfd9b22449ba95ba52818bac30d3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.611335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dd934-3d08-4a4b-ac5b-3b92afc10c44 None admin] RBAC: Authorizing `identity:get_user(user_id=826adfd9b22449ba95ba52818bac30d3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.612772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0dd934-3d08-4a4b-ac5b-3b92afc10c44 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.618722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 59/120] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/users/826adfd9b22449ba95ba52818bac30d3 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:22.618722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dd934-3d08-4a4b-ac5b-3b92afc10c44 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.627066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9bd29-1b87-43b3-9b5e-d6740f6c4823 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.661444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d598f6-5678-40e6-bad6-2b67da562b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.662075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d598f6-5678-40e6-bad6-2b67da562b15 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.662360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d598f6-5678-40e6-bad6-2b67da562b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.662635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d598f6-5678-40e6-bad6-2b67da562b15 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.663770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d598f6-5678-40e6-bad6-2b67da562b15 None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.665213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d598f6-5678-40e6-bad6-2b67da562b15 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.667032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 62/121] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:22.672035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d598f6-5678-40e6-bad6-2b67da562b15 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.727782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0dd934-3d08-4a4b-ac5b-3b92afc10c44 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.752028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:22.763248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8014ee9-2025-4f3d-951b-70a3763c56bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.763891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8014ee9-2025-4f3d-951b-70a3763c56bf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.764182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8014ee9-2025-4f3d-951b-70a3763c56bf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.764464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8014ee9-2025-4f3d-951b-70a3763c56bf None admin] PATH_INFO: `/v3/users/826adfd9b22449ba95ba52818bac30d3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.765597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8014ee9-2025-4f3d-951b-70a3763c56bf None admin] RBAC: Authorizing `identity:get_user(user_id=826adfd9b22449ba95ba52818bac30d3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.767074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8014ee9-2025-4f3d-951b-70a3763c56bf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.768673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 60/122] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/users/826adfd9b22449ba95ba52818bac30d3 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:22.773324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8014ee9-2025-4f3d-951b-70a3763c56bf None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:22.789640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d598f6-5678-40e6-bad6-2b67da562b15 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.801615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:22.826695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f435b-d689-4415-b649-5091629b575b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.827338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f435b-d689-4415-b649-5091629b575b None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.827627 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f435b-d689-4415-b649-5091629b575b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.827906 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f435b-d689-4415-b649-5091629b575b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.878667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8014ee9-2025-4f3d-951b-70a3763c56bf None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:22.898955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f435b-d689-4415-b649-5091629b575b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.900964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f435b-d689-4415-b649-5091629b575b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.916856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a97285-1010-494a-920b-676eeaed320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:22.917466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a97285-1010-494a-920b-676eeaed320d None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:22.917760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a97285-1010-494a-920b-676eeaed320d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:22.918633 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a97285-1010-494a-920b-676eeaed320d None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2/users/85d813f29e9b43528ce7905c0d7cb0e8/roles/542d76fcc5884c82aa0a0375961f29c2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:22.920817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a97285-1010-494a-920b-676eeaed320d None admin] RBAC: Authorizing `identity:create_grant(project_id=1634123c90c5493296570578db87c5c2, user_id=85d813f29e9b43528ce7905c0d7cb0e8, role_id=542d76fcc5884c82aa0a0375961f29c2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:22.922276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a97285-1010-494a-920b-676eeaed320d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:22.937022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 61/123] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:22 2022] PUT /identity/v3/projects/1634123c90c5493296570578db87c5c2/users/85d813f29e9b43528ce7905c0d7cb0e8/roles/542d76fcc5884c82aa0a0375961f29c2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:22.947263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a97285-1010-494a-920b-676eeaed320d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:23.002047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:23.039529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:23.042050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 63/124] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:23.048317 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f435b-d689-4415-b649-5091629b575b None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:23.057002 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a97285-1010-494a-920b-676eeaed320d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:23.104133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c6cb8-59c0-413a-8bd9-d917ea12243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:23.104769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c6cb8-59c0-413a-8bd9-d917ea12243d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:23.105060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c6cb8-59c0-413a-8bd9-d917ea12243d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:23.105340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c6cb8-59c0-413a-8bd9-d917ea12243d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:23.192710 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c6cb8-59c0-413a-8bd9-d917ea12243d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:23.193087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f435b-d689-4415-b649-5091629b575b None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:23.194835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c6cb8-59c0-413a-8bd9-d917ea12243d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:23.236572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8298fe-c4c6-4f82-b22f-0bd6a8e8a687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:23.237276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8298fe-c4c6-4f82-b22f-0bd6a8e8a687 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:23.237567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8298fe-c4c6-4f82-b22f-0bd6a8e8a687 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:23.237848 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8298fe-c4c6-4f82-b22f-0bd6a8e8a687 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:23.239452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8298fe-c4c6-4f82-b22f-0bd6a8e8a687 None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:23.241009 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8298fe-c4c6-4f82-b22f-0bd6a8e8a687 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:23.243806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 64/125] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:23 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:23.250104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8298fe-c4c6-4f82-b22f-0bd6a8e8a687 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:23.293887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:23.294513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:23.314924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 62/126] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:23.321217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c6cb8-59c0-413a-8bd9-d917ea12243d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:23.368018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8298fe-c4c6-4f82-b22f-0bd6a8e8a687 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:23.412860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:23.413478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:23.413772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:23.414426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:23.415218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] RBAC: Adding query filter params (role.id=78ff1dcdc39743048089711fcfc4d3a4, scope.project.id=1634123c90c5493296570578db87c5c2, user.id=826adfd9b22449ba95ba52818bac30d3) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:23.415540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:23.417630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:23.425057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 65/127] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:23 2022] GET /identity/v3/role_assignments?user.id=826adfd9b22449ba95ba52818bac30d3&scope.project.id=1634123c90c5493296570578db87c5c2&role.id=78ff1dcdc39743048089711fcfc4d3a4 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:23.444735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c6cb8-59c0-413a-8bd9-d917ea12243d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:23.484727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb99f4e-962f-4eba-91ee-18a733062d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:23.485637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99f4e-962f-4eba-91ee-18a733062d8f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:23.486174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99f4e-962f-4eba-91ee-18a733062d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:23.486604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb99f4e-962f-4eba-91ee-18a733062d8f None admin] PATH_INFO: `/v3/projects/4f46937fc7374b7fb2f1dbd0b4ea65cb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:23.488080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb99f4e-962f-4eba-91ee-18a733062d8f None admin] RBAC: Authorizing `identity:get_project(project_id=4f46937fc7374b7fb2f1dbd0b4ea65cb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:23.489768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb99f4e-962f-4eba-91ee-18a733062d8f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:23.492175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 63/128] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:23 2022] GET /identity/v3/projects/4f46937fc7374b7fb2f1dbd0b4ea65cb => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:23.498763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:23.614265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9432b-0e2c-4eaf-8851-313d8a67a1f9 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:23.616952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:23.654655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e882c2d-2d0b-46d1-930f-8e97bd2d0450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:23.655338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e882c2d-2d0b-46d1-930f-8e97bd2d0450 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:23.655631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e882c2d-2d0b-46d1-930f-8e97bd2d0450 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:23.655913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e882c2d-2d0b-46d1-930f-8e97bd2d0450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:23.656670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e882c2d-2d0b-46d1-930f-8e97bd2d0450 None admin] RBAC: Adding query filter params (role.id=54e93caca0fb4c59a01ae64a0ed3a610, scope.project.id=4f46937fc7374b7fb2f1dbd0b4ea65cb, user.id=826adfd9b22449ba95ba52818bac30d3) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:23.656990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e882c2d-2d0b-46d1-930f-8e97bd2d0450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:23.658608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e882c2d-2d0b-46d1-930f-8e97bd2d0450 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:23.665404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 66/129] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:23 2022] GET /identity/v3/role_assignments?user.id=826adfd9b22449ba95ba52818bac30d3&scope.project.id=4f46937fc7374b7fb2f1dbd0b4ea65cb&role.id=54e93caca0fb4c59a01ae64a0ed3a610 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:24.038241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4436-4164-44bd-a967-ed4bb830b5e6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:24.038559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4436-4164-44bd-a967-ed4bb830b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:24.038842 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f4436-4164-44bd-a967-ed4bb830b5e6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:24.040230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 64/130] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:24.046175 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833a2c-4b6f-4371-b284-44de5813a5b2 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:24.046554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833a2c-4b6f-4371-b284-44de5813a5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:24.046904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60833a2c-4b6f-4371-b284-44de5813a5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:24.064973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-60833a2c-4b6f-4371-b284-44de5813a5b2 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:24.066170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-60833a2c-4b6f-4371-b284-44de5813a5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:24.136379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 67/131] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:24.407338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f4436-4164-44bd-a967-ed4bb830b5e6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:24.462181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:24.507114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f4436-4164-44bd-a967-ed4bb830b5e6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:24.540790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39d043-f6dc-40f9-8791-1b418665b664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:24.541421 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39d043-f6dc-40f9-8791-1b418665b664 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:24.541707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39d043-f6dc-40f9-8791-1b418665b664 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:24.541988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39d043-f6dc-40f9-8791-1b418665b664 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:24.543281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39d043-f6dc-40f9-8791-1b418665b664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:24.544917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39d043-f6dc-40f9-8791-1b418665b664 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:24.546495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 65/132] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:24.552123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60833a2c-4b6f-4371-b284-44de5813a5b2 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:24.645882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:24.650271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60833a2c-4b6f-4371-b284-44de5813a5b2 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:24.685046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64474f24-3ba2-44c6-b9a4-d82a54981cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:24.685586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64474f24-3ba2-44c6-b9a4-d82a54981cbf None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:24.685874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64474f24-3ba2-44c6-b9a4-d82a54981cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:24.686154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64474f24-3ba2-44c6-b9a4-d82a54981cbf None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:24.686817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64474f24-3ba2-44c6-b9a4-d82a54981cbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:24.688321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64474f24-3ba2-44c6-b9a4-d82a54981cbf None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:24.709227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 68/133] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:24 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:25.174982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90feb1c-42c5-4bae-a5e3-ab17cba97a3c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:25.175344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90feb1c-42c5-4bae-a5e3-ab17cba97a3c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:25.175589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90feb1c-42c5-4bae-a5e3-ab17cba97a3c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:25.176938 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 66/134] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:25.182258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2105e9-0c02-4ac6-91d6-d4b8e428586a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:25.182550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2105e9-0c02-4ac6-91d6-d4b8e428586a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:25.182834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2105e9-0c02-4ac6-91d6-d4b8e428586a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:25.201694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-de2105e9-0c02-4ac6-91d6-d4b8e428586a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:25.202901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-de2105e9-0c02-4ac6-91d6-d4b8e428586a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:25.273394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 69/135] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:25.545388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90feb1c-42c5-4bae-a5e3-ab17cba97a3c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:25.679105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90feb1c-42c5-4bae-a5e3-ab17cba97a3c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:25.711485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:25.723087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350a54c-aec6-4106-80a8-dd5a6d10fa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:25.723087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350a54c-aec6-4106-80a8-dd5a6d10fa9c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:25.723087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350a54c-aec6-4106-80a8-dd5a6d10fa9c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:25.723087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350a54c-aec6-4106-80a8-dd5a6d10fa9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:25.737047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350a54c-aec6-4106-80a8-dd5a6d10fa9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4350a54c-aec6-4106-80a8-dd5a6d10fa9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:25.742268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:25.743139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:25.743909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 67/136] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:25.750194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2105e9-0c02-4ac6-91d6-d4b8e428586a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:25.862937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:25.890312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2105e9-0c02-4ac6-91d6-d4b8e428586a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:25.947761 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:25.948384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:25.948665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:25.948945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:25.949919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:25.950335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:25.952999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:25.972354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 70/137] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:25.974004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350a54c-aec6-4106-80a8-dd5a6d10fa9c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:26.091638 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350a54c-aec6-4106-80a8-dd5a6d10fa9c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:26.127347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a70d9-c578-4be7-ab9e-2dfc4d1863bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:26.127905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a70d9-c578-4be7-ab9e-2dfc4d1863bc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:26.128189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a70d9-c578-4be7-ab9e-2dfc4d1863bc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:26.128467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a70d9-c578-4be7-ab9e-2dfc4d1863bc None admin] PATH_INFO: `/v3/users/85d813f29e9b43528ce7905c0d7cb0e8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:26.129838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a70d9-c578-4be7-ab9e-2dfc4d1863bc None admin] RBAC: Authorizing `identity:get_user(user_id=85d813f29e9b43528ce7905c0d7cb0e8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:26.131613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a70d9-c578-4be7-ab9e-2dfc4d1863bc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:26.138740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 68/138] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:25 2022] GET /identity/v3/users/85d813f29e9b43528ce7905c0d7cb0e8 => generated 289 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:26.140125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:26.276316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13881285-1274-40a3-a858-c2a28481aac5 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:26.331794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2edca-88f9-4309-86df-e9e3ffccc2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:26.332422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2edca-88f9-4309-86df-e9e3ffccc2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:26.332702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2edca-88f9-4309-86df-e9e3ffccc2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:26.332977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2edca-88f9-4309-86df-e9e3ffccc2b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:26.363457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:26.425388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2edca-88f9-4309-86df-e9e3ffccc2b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:26.427216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2edca-88f9-4309-86df-e9e3ffccc2b1 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:26.530729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 71/139] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 12:59:26.536604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a70d9-c578-4be7-ab9e-2dfc4d1863bc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:26.592613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:26.643264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a70d9-c578-4be7-ab9e-2dfc4d1863bc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:26.678096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94584b7-5687-476d-892e-4a512a5c2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:26.678659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94584b7-5687-476d-892e-4a512a5c2ffb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:26.678971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94584b7-5687-476d-892e-4a512a5c2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:26.679361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94584b7-5687-476d-892e-4a512a5c2ffb None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:26.680505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94584b7-5687-476d-892e-4a512a5c2ffb None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:26.682051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94584b7-5687-476d-892e-4a512a5c2ffb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:26.684279 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 69/140] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:26 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:26.690490 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2edca-88f9-4309-86df-e9e3ffccc2b1 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:26.739175 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:26.794847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2edca-88f9-4309-86df-e9e3ffccc2b1 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:26.831950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63172283-05e3-4481-aec6-6a5df2f8f305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:26.832566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63172283-05e3-4481-aec6-6a5df2f8f305 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:26.832847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63172283-05e3-4481-aec6-6a5df2f8f305 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:26.833121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63172283-05e3-4481-aec6-6a5df2f8f305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:26.833849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63172283-05e3-4481-aec6-6a5df2f8f305 None admin] RBAC: Adding query filter params (role.id=542d76fcc5884c82aa0a0375961f29c2, scope.project.id=1634123c90c5493296570578db87c5c2, user.id=85d813f29e9b43528ce7905c0d7cb0e8) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:26.834162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63172283-05e3-4481-aec6-6a5df2f8f305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:26.835654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63172283-05e3-4481-aec6-6a5df2f8f305 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:26.842545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 72/141] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:26 2022] GET /identity/v3/role_assignments?user.id=85d813f29e9b43528ce7905c0d7cb0e8&scope.project.id=1634123c90c5493296570578db87c5c2&role.id=542d76fcc5884c82aa0a0375961f29c2 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:27.083550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805f90-10aa-42ef-88b9-2ea332bcf02c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.084072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805f90-10aa-42ef-88b9-2ea332bcf02c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.084370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b805f90-10aa-42ef-88b9-2ea332bcf02c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.085731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 70/142] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:27.091645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4fb48-a7e7-487f-b383-69aa56bb4bb9 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.091934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4fb48-a7e7-487f-b383-69aa56bb4bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.092210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4fb48-a7e7-487f-b383-69aa56bb4bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.111886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-60b4fb48-a7e7-487f-b383-69aa56bb4bb9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:27.113074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-60b4fb48-a7e7-487f-b383-69aa56bb4bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:27.192938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 73/143] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:27.252777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de12b4-7246-439c-8efd-dcbc74875cac None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.253059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de12b4-7246-439c-8efd-dcbc74875cac None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.253335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de12b4-7246-439c-8efd-dcbc74875cac None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.254824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 71/144] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:27.260062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba366-bf13-414d-a910-2ca090c5bb53 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.260358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba366-bf13-414d-a910-2ca090c5bb53 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.260633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba366-bf13-414d-a910-2ca090c5bb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.279248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-54aba366-bf13-414d-a910-2ca090c5bb53 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:27.280425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-54aba366-bf13-414d-a910-2ca090c5bb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:27.316283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:27.353998 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 74/145] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:27.435608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4de1-2e88-43b0-af43-7d1925857695 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.435920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4de1-2e88-43b0-af43-7d1925857695 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.436173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed4de1-2e88-43b0-af43-7d1925857695 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.437642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 72/146] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:27.442755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11004-d24e-4183-bc14-30cc73ee2e2c None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.443361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11004-d24e-4183-bc14-30cc73ee2e2c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.443637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11004-d24e-4183-bc14-30cc73ee2e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.462015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a9a11004-d24e-4183-bc14-30cc73ee2e2c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:27.463201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a9a11004-d24e-4183-bc14-30cc73ee2e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:27.467043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed4de1-2e88-43b0-af43-7d1925857695 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:27.533726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 75/147] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:27.570933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed4de1-2e88-43b0-af43-7d1925857695 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:27.588597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:27.606870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa79b0a-f5b9-476b-b65d-eeada3d5fe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:27.607492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79b0a-f5b9-476b-b65d-eeada3d5fe2e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.607751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79b0a-f5b9-476b-b65d-eeada3d5fe2e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.608032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79b0a-f5b9-476b-b65d-eeada3d5fe2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.618098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa79b0a-f5b9-476b-b65d-eeada3d5fe2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-1fa79b0a-f5b9-476b-b65d-eeada3d5fe2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:27.623040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:27.623970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:27.624779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:27.625562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a11004-d24e-4183-bc14-30cc73ee2e2c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:27.625644 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 73/148] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 12:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:27.629778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa79b0a-f5b9-476b-b65d-eeada3d5fe2e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:27.730135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:27.734564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a11004-d24e-4183-bc14-30cc73ee2e2c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:27.742523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa79b0a-f5b9-476b-b65d-eeada3d5fe2e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:27.774731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafa303-6785-420e-9d33-99132f65583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:27.775443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafa303-6785-420e-9d33-99132f65583b None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.775703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafa303-6785-420e-9d33-99132f65583b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.775979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafa303-6785-420e-9d33-99132f65583b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.781704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:27.782530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.783022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.783514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.784362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:27.784843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:27.786399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:27.786742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cafa303-6785-420e-9d33-99132f65583b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-0cafa303-6785-420e-9d33-99132f65583b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:27.791675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:27.792528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:27.793872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 76/149] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 12:59:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:27.798906 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafa303-6785-420e-9d33-99132f65583b None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:27.801800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 74/150] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:27.808039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:27.849549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:27.901093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafa303-6785-420e-9d33-99132f65583b None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:27.906752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b8b0d-d039-4b1d-916a-f45a2b7b68dd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:27.936186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:27.936788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.937065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.937336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.937989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:27.938294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:27.939701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:27.941073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fc622-1480-4781-80f2-f0d08b60090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:27.941707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc622-1480-4781-80f2-f0d08b60090a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:27.942005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc622-1480-4781-80f2-f0d08b60090a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:27.942283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72fc622-1480-4781-80f2-f0d08b60090a None admin] PATH_INFO: `/v3/users/8d2598fc20b04fcc929813401826a211` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:27.948494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 77/151] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 12:59:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:27.949934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:27.956675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fc622-1480-4781-80f2-f0d08b60090a None admin] RBAC: Authorizing `identity:get_user(user_id=8d2598fc20b04fcc929813401826a211)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:27.958128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72fc622-1480-4781-80f2-f0d08b60090a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:27.959814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 75/152] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:27 2022] GET /identity/v3/users/8d2598fc20b04fcc929813401826a211 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:27.960919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fc622-1480-4781-80f2-f0d08b60090a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:27.977144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:28.053389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a234fb-7d0c-429b-906c-a28b0fe843f8 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:28.061030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:28.066195 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72fc622-1480-4781-80f2-f0d08b60090a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:28.088814 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e35a-54d9-4ade-a144-933023ff4054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:28.089445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e35a-54d9-4ade-a144-933023ff4054 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:28.090242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e35a-54d9-4ade-a144-933023ff4054 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:28.090956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52e35a-54d9-4ade-a144-933023ff4054 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:28.092096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52e35a-54d9-4ade-a144-933023ff4054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:28.093539 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52e35a-54d9-4ade-a144-933023ff4054 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:28.095112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 78/153] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:28.100009 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e35a-54d9-4ade-a144-933023ff4054 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:28.100667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dde6b-019b-49a8-9fff-e32ec3867ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:28.101299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dde6b-019b-49a8-9fff-e32ec3867ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:28.101585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dde6b-019b-49a8-9fff-e32ec3867ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:28.101865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dde6b-019b-49a8-9fff-e32ec3867ec4 None admin] PATH_INFO: `/v3/users/8d2598fc20b04fcc929813401826a211` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:28.103141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dde6b-019b-49a8-9fff-e32ec3867ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=8d2598fc20b04fcc929813401826a211)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:28.104585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dde6b-019b-49a8-9fff-e32ec3867ec4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:28.106974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 76/154] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:27 2022] GET /identity/v3/users/8d2598fc20b04fcc929813401826a211 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:28.107739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dde6b-019b-49a8-9fff-e32ec3867ec4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:28.190531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:28.201491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52e35a-54d9-4ade-a144-933023ff4054 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:28.207869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dde6b-019b-49a8-9fff-e32ec3867ec4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:28.234661 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d2c01-30d8-4048-afb2-a8fc47b6ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:28.235321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d2c01-30d8-4048-afb2-a8fc47b6ddf4 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:28.235590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d2c01-30d8-4048-afb2-a8fc47b6ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:28.235860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d2c01-30d8-4048-afb2-a8fc47b6ddf4 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:28.236959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d2c01-30d8-4048-afb2-a8fc47b6ddf4 None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:28.238390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d2c01-30d8-4048-afb2-a8fc47b6ddf4 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:28.240027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 79/155] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:28 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:28.240862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d2c01-30d8-4048-afb2-a8fc47b6ddf4 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:28.244709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552e37-f1fa-4a66-afcc-de54455b838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:28.245315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552e37-f1fa-4a66-afcc-de54455b838e None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:28.245597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552e37-f1fa-4a66-afcc-de54455b838e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:28.245875 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552e37-f1fa-4a66-afcc-de54455b838e None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:28.246546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e552e37-f1fa-4a66-afcc-de54455b838e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:28.248038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e552e37-f1fa-4a66-afcc-de54455b838e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:28.271610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 77/156] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:28 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:28.276080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552e37-f1fa-4a66-afcc-de54455b838e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:28.306271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:28.357208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d2c01-30d8-4048-afb2-a8fc47b6ddf4 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:28.396331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178474aa-5a62-4dfd-a512-33945ef967ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:28.396977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178474aa-5a62-4dfd-a512-33945ef967ee None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:28.397259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178474aa-5a62-4dfd-a512-33945ef967ee None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:28.397355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552e37-f1fa-4a66-afcc-de54455b838e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:28.397588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178474aa-5a62-4dfd-a512-33945ef967ee None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:28.398688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178474aa-5a62-4dfd-a512-33945ef967ee None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:28.400196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178474aa-5a62-4dfd-a512-33945ef967ee None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:28.402070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 80/157] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:28 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:28.408398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178474aa-5a62-4dfd-a512-33945ef967ee None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:28.437564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e51d7-dfd3-45e6-91f5-6138df113ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:28.438204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:28.438313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e51d7-dfd3-45e6-91f5-6138df113ba3 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:28.438528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e51d7-dfd3-45e6-91f5-6138df113ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:28.438814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e51d7-dfd3-45e6-91f5-6138df113ba3 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/users/8d2598fc20b04fcc929813401826a211/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:28.441100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e51d7-dfd3-45e6-91f5-6138df113ba3 None admin] RBAC: Authorizing `identity:create_grant(project_id=97d9afe79c47436899dad4fbd1cd7bfd, user_id=8d2598fc20b04fcc929813401826a211, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:28.442521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e51d7-dfd3-45e6-91f5-6138df113ba3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:28.458188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 78/158] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:28 2022] PUT /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/users/8d2598fc20b04fcc929813401826a211/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:28.528291 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178474aa-5a62-4dfd-a512-33945ef967ee None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:28.566635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac631e23-3472-4b6b-81fe-31d487eb2bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:28.567443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631e23-3472-4b6b-81fe-31d487eb2bc4 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:28.567729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631e23-3472-4b6b-81fe-31d487eb2bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:28.568004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631e23-3472-4b6b-81fe-31d487eb2bc4 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/users/8d2598fc20b04fcc929813401826a211/roles/78ff1dcdc39743048089711fcfc4d3a4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:28.570292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac631e23-3472-4b6b-81fe-31d487eb2bc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=97d9afe79c47436899dad4fbd1cd7bfd, user_id=8d2598fc20b04fcc929813401826a211, role_id=78ff1dcdc39743048089711fcfc4d3a4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:28.571847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac631e23-3472-4b6b-81fe-31d487eb2bc4 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:28.589055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 81/159] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:28 2022] PUT /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/users/8d2598fc20b04fcc929813401826a211/roles/78ff1dcdc39743048089711fcfc4d3a4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 12:59:30.625490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c7f9a-52cc-4680-98f5-c573cc48b69d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:30.626046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c7f9a-52cc-4680-98f5-c573cc48b69d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:30.626046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c7f9a-52cc-4680-98f5-c573cc48b69d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:30.627664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 79/160] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:30.633583 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171e22c-e4f7-44ed-8dd6-146f5c9c9d18 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:30.634114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171e22c-e4f7-44ed-8dd6-146f5c9c9d18 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:30.634114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171e22c-e4f7-44ed-8dd6-146f5c9c9d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:30.654514 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0171e22c-e4f7-44ed-8dd6-146f5c9c9d18 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:30.655735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0171e22c-e4f7-44ed-8dd6-146f5c9c9d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:30.659539 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed617c-9adc-42cc-afac-622979328fd2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:30.660085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed617c-9adc-42cc-afac-622979328fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:30.660584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed617c-9adc-42cc-afac-622979328fd2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:30.662199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 80/161] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:59:30.668439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46970f-434e-4950-83d6-d7aa6152ce67 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:30.669078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46970f-434e-4950-83d6-d7aa6152ce67 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:30.669432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46970f-434e-4950-83d6-d7aa6152ce67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:30.692580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8a46970f-434e-4950-83d6-d7aa6152ce67 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:30.694363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8a46970f-434e-4950-83d6-d7aa6152ce67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:30.743080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 82/162] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:30.747908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3344e-e56d-4a3a-9955-5b16ecbc65e3 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:30.748200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3344e-e56d-4a3a-9955-5b16ecbc65e3 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:30.748466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3344e-e56d-4a3a-9955-5b16ecbc65e3 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:30.749918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 83/163] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:59:30.754839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38120046-899c-41f8-9742-34ed234af9fc None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:30.755295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38120046-899c-41f8-9742-34ed234af9fc None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:30.755711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38120046-899c-41f8-9742-34ed234af9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:30.775364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-38120046-899c-41f8-9742-34ed234af9fc None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:30.776739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-38120046-899c-41f8-9742-34ed234af9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:30.781954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 81/164] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:30.789658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:30.819689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888af3-a35d-4ef7-b190-bbdae2e3a997 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:30.820073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888af3-a35d-4ef7-b190-bbdae2e3a997 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:30.820417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40888af3-a35d-4ef7-b190-bbdae2e3a997 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:30.821826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 82/165] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:59:30.827805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381729-4124-4175-8948-e7bd31eede31 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:30.828219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381729-4124-4175-8948-e7bd31eede31 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:30.828594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a381729-4124-4175-8948-e7bd31eede31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:30.849951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9a381729-4124-4175-8948-e7bd31eede31 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:30.851289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9a381729-4124-4175-8948-e7bd31eede31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:30.852622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 84/166] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:30.864236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:30.926118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 83/167] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:31.013772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38120046-899c-41f8-9742-34ed234af9fc None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.057901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a381729-4124-4175-8948-e7bd31eede31 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.128083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38120046-899c-41f8-9742-34ed234af9fc None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.167131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0a141-3ca2-45f0-90b6-986faddf2fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.168366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0a141-3ca2-45f0-90b6-986faddf2fef None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.168655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0a141-3ca2-45f0-90b6-986faddf2fef None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.169026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0a141-3ca2-45f0-90b6-986faddf2fef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.178223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a381729-4124-4175-8948-e7bd31eede31 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.179646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0a141-3ca2-45f0-90b6-986faddf2fef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-43c0a141-3ca2-45f0-90b6-986faddf2fef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:31.185283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:31.186165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:31.186939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 85/168] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:31.188162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0a141-3ca2-45f0-90b6-986faddf2fef None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.197022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:31.222859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e8a8c-cc9e-4fba-b8f9-8fd3ae4c1c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.223525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8a8c-cc9e-4fba-b8f9-8fd3ae4c1c9c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.223803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8a8c-cc9e-4fba-b8f9-8fd3ae4c1c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.224088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e8a8c-cc9e-4fba-b8f9-8fd3ae4c1c9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.234609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e8a8c-cc9e-4fba-b8f9-8fd3ae4c1c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-943e8a8c-cc9e-4fba-b8f9-8fd3ae4c1c9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:31.239395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:31.240411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:31.241180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 84/169] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:31.241836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e8a8c-cc9e-4fba-b8f9-8fd3ae4c1c9c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.246764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:31.290515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0a141-3ca2-45f0-90b6-986faddf2fef None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.323741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f544151-1a98-4507-8ea8-fb3ff70587e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.334905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544151-1a98-4507-8ea8-fb3ff70587e2 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.335213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544151-1a98-4507-8ea8-fb3ff70587e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.335489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f544151-1a98-4507-8ea8-fb3ff70587e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.336628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f544151-1a98-4507-8ea8-fb3ff70587e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:31.338088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f544151-1a98-4507-8ea8-fb3ff70587e2 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:31.339689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 86/170] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:31.342804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e8a8c-cc9e-4fba-b8f9-8fd3ae4c1c9c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.343747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f544151-1a98-4507-8ea8-fb3ff70587e2 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.376900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.377499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.377776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.378045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.378703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:31.379003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:31.380389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:31.389496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 85/171] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:31.390604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.422122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:31.448055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f544151-1a98-4507-8ea8-fb3ff70587e2 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.452142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:31.482683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cccfc-6603-4a33-95aa-1f04b5cd9de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.483334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cccfc-6603-4a33-95aa-1f04b5cd9de0 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.483617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cccfc-6603-4a33-95aa-1f04b5cd9de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.483888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cccfc-6603-4a33-95aa-1f04b5cd9de0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.490410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46f8da-911a-4541-b635-5338a8c945be None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.493665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cccfc-6603-4a33-95aa-1f04b5cd9de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-421cccfc-6603-4a33-95aa-1f04b5cd9de0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:31.499482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:31.500376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:31.501185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 87/172] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:31.501829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cccfc-6603-4a33-95aa-1f04b5cd9de0 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.526950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.528072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.528358 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.528633 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.529301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:31.529602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:31.530964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:31.538710 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 86/173] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:31.539510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.606453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cccfc-6603-4a33-95aa-1f04b5cd9de0 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.638096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb03506-89a6-4bfc-b25b-fa323e905805 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.642179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c82a00-c35d-49e4-91fa-6331803b3e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.642780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c82a00-c35d-49e4-91fa-6331803b3e69 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.643079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c82a00-c35d-49e4-91fa-6331803b3e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.643352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c82a00-c35d-49e4-91fa-6331803b3e69 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.643987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c82a00-c35d-49e4-91fa-6331803b3e69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:31.645432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c82a00-c35d-49e4-91fa-6331803b3e69 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:31.653950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:31.664408 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:31.675562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 88/174] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:31 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:31.675562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c82a00-c35d-49e4-91fa-6331803b3e69 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.685986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dddff-d37c-40bc-9a57-8c0bd42c5bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.686762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dddff-d37c-40bc-9a57-8c0bd42c5bdb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.687075 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dddff-d37c-40bc-9a57-8c0bd42c5bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.687372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47dddff-d37c-40bc-9a57-8c0bd42c5bdb None admin] PATH_INFO: `/v3/users/a10bfa156dbb46d3863840e801b7b0fe` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.705248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dddff-d37c-40bc-9a57-8c0bd42c5bdb None admin] RBAC: Authorizing `identity:get_user(user_id=a10bfa156dbb46d3863840e801b7b0fe)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:31.706938 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47dddff-d37c-40bc-9a57-8c0bd42c5bdb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:31.709391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 87/175] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/users/a10bfa156dbb46d3863840e801b7b0fe => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:31.710741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dddff-d37c-40bc-9a57-8c0bd42c5bdb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.790833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c82a00-c35d-49e4-91fa-6331803b3e69 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.832256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47dddff-d37c-40bc-9a57-8c0bd42c5bdb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:31.832460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.833011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.833299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.833578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.834266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:31.834577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:31.836013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:31.846077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 89/176] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:31.846268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.881754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce39d7-4e21-4e50-83d2-4d31d6fc8729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:31.882676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce39d7-4e21-4e50-83d2-4d31d6fc8729 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:31.882962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce39d7-4e21-4e50-83d2-4d31d6fc8729 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:31.883287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce39d7-4e21-4e50-83d2-4d31d6fc8729 None admin] PATH_INFO: `/v3/users/8d2598fc20b04fcc929813401826a211` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:31.884411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce39d7-4e21-4e50-83d2-4d31d6fc8729 None admin] RBAC: Authorizing `identity:get_user(user_id=8d2598fc20b04fcc929813401826a211)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:31.885960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce39d7-4e21-4e50-83d2-4d31d6fc8729 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:31.893589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 88/177] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/users/8d2598fc20b04fcc929813401826a211 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:31.893589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce39d7-4e21-4e50-83d2-4d31d6fc8729 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:31.911281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:31.966207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:31.976516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5646de9-f563-4e7c-9a2a-976669688d4a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:32.025974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b413c4-2ec9-4c98-ad21-defba249a42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:32.027005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b413c4-2ec9-4c98-ad21-defba249a42d None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:32.027766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce39d7-4e21-4e50-83d2-4d31d6fc8729 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:32.028016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b413c4-2ec9-4c98-ad21-defba249a42d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:32.028216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b413c4-2ec9-4c98-ad21-defba249a42d None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:32.029480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b413c4-2ec9-4c98-ad21-defba249a42d None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:32.031708 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b413c4-2ec9-4c98-ad21-defba249a42d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:32.035869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 90/178] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:32.035869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b413c4-2ec9-4c98-ad21-defba249a42d None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:32.081831 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce73de-1145-4241-b9d8-b61ebfcab4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:32.083678 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce73de-1145-4241-b9d8-b61ebfcab4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:32.084230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce73de-1145-4241-b9d8-b61ebfcab4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:32.084691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce73de-1145-4241-b9d8-b61ebfcab4b8 None admin] PATH_INFO: `/v3/users/8d2598fc20b04fcc929813401826a211` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:32.086156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce73de-1145-4241-b9d8-b61ebfcab4b8 None admin] RBAC: Authorizing `identity:get_user(user_id=8d2598fc20b04fcc929813401826a211)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:32.088522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce73de-1145-4241-b9d8-b61ebfcab4b8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:32.092190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 89/179] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:31 2022] GET /identity/v3/users/8d2598fc20b04fcc929813401826a211 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:32.093223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce73de-1145-4241-b9d8-b61ebfcab4b8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:32.167348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b413c4-2ec9-4c98-ad21-defba249a42d None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:32.193663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:32.212107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649366d9-4d8f-4598-baea-66691a0169d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:32.212825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649366d9-4d8f-4598-baea-66691a0169d2 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:32.213182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649366d9-4d8f-4598-baea-66691a0169d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:32.213532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649366d9-4d8f-4598-baea-66691a0169d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:32.234768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce73de-1145-4241-b9d8-b61ebfcab4b8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:32.262640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:32.282568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1b1a1-fc6b-46a5-bba1-565e1515b794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:32.283827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1b1a1-fc6b-46a5-bba1-565e1515b794 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:32.284397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1b1a1-fc6b-46a5-bba1-565e1515b794 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:32.284855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c1b1a1-fc6b-46a5-bba1-565e1515b794 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/users/a10bfa156dbb46d3863840e801b7b0fe/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:32.288073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1b1a1-fc6b-46a5-bba1-565e1515b794 None admin] RBAC: Authorizing `identity:create_grant(project_id=97d9afe79c47436899dad4fbd1cd7bfd, user_id=a10bfa156dbb46d3863840e801b7b0fe, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:32.290809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c1b1a1-fc6b-46a5-bba1-565e1515b794 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:32.301666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649366d9-4d8f-4598-baea-66691a0169d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:32.303655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649366d9-4d8f-4598-baea-66691a0169d2 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:32.309738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 90/180] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:32 2022] PUT /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/users/a10bfa156dbb46d3863840e801b7b0fe/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:32.316349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1b1a1-fc6b-46a5-bba1-565e1515b794 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:32.439103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c1b1a1-fc6b-46a5-bba1-565e1515b794 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:32.503783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 91/181] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:32.509663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649366d9-4d8f-4598-baea-66691a0169d2 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:32.534991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:32.535689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75f0a6-2fa5-469a-a16a-c3dc3b7b8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:32.536324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75f0a6-2fa5-469a-a16a-c3dc3b7b8113 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:32.536611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75f0a6-2fa5-469a-a16a-c3dc3b7b8113 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:32.536891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75f0a6-2fa5-469a-a16a-c3dc3b7b8113 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:32.607294 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:32.631819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649366d9-4d8f-4598-baea-66691a0169d2 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:32.633253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75f0a6-2fa5-469a-a16a-c3dc3b7b8113 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:32.634744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75f0a6-2fa5-469a-a16a-c3dc3b7b8113 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:32.679663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae346b-a216-4718-9d3f-6ed444680603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:32.680400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae346b-a216-4718-9d3f-6ed444680603 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:32.680820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae346b-a216-4718-9d3f-6ed444680603 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:32.681105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae346b-a216-4718-9d3f-6ed444680603 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:32.682947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae346b-a216-4718-9d3f-6ed444680603 None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:32.684639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae346b-a216-4718-9d3f-6ed444680603 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:32.686858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 92/182] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:32 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:32.693617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae346b-a216-4718-9d3f-6ed444680603 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:32.800100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:32.815101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae346b-a216-4718-9d3f-6ed444680603 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:32.829966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 91/183] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:32.835624 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75f0a6-2fa5-469a-a16a-c3dc3b7b8113 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:32.855902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:32.856723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:32.857021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:32.857302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:32.858074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd, user.id=8d2598fc20b04fcc929813401826a211) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:32.858389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:32.860429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:32.867174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 93/184] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:32 2022] GET /identity/v3/role_assignments?user.id=8d2598fc20b04fcc929813401826a211&scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd&role.id=11b74890c7484c6088dd78c106de8be4 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:32.994128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:33.020934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75f0a6-2fa5-469a-a16a-c3dc3b7b8113 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:33.074796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f1923-89f4-4fc9-8377-dec49a738ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:33.075479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1923-89f4-4fc9-8377-dec49a738ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:33.075764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1923-89f4-4fc9-8377-dec49a738ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:33.076044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f1923-89f4-4fc9-8377-dec49a738ea7 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:33.079730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f1923-89f4-4fc9-8377-dec49a738ea7 None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:33.081193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f1923-89f4-4fc9-8377-dec49a738ea7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:33.082835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 92/185] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:32 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:33.088635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:33.244348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e155a0-7946-45e9-92e4-39e6b53632a0 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:33.287742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a472122-6c1b-4cdc-b3c2-fa260aea533c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:33.288414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472122-6c1b-4cdc-b3c2-fa260aea533c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:33.288703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472122-6c1b-4cdc-b3c2-fa260aea533c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:33.288981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a472122-6c1b-4cdc-b3c2-fa260aea533c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:33.289716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a472122-6c1b-4cdc-b3c2-fa260aea533c None admin] RBAC: Adding query filter params (role.id=78ff1dcdc39743048089711fcfc4d3a4, scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd, user.id=8d2598fc20b04fcc929813401826a211) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:33.290031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a472122-6c1b-4cdc-b3c2-fa260aea533c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:33.291563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a472122-6c1b-4cdc-b3c2-fa260aea533c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:33.298923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 94/186] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:33 2022] GET /identity/v3/role_assignments?user.id=8d2598fc20b04fcc929813401826a211&scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd&role.id=78ff1dcdc39743048089711fcfc4d3a4 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:34.168514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458761-935c-4b8d-aae0-90e0b6b668a0 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.169038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458761-935c-4b8d-aae0-90e0b6b668a0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.169038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458761-935c-4b8d-aae0-90e0b6b668a0 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.170536 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 93/187] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:34.175499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3076231-5959-4fc2-b204-ca5d564cac41 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.175900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3076231-5959-4fc2-b204-ca5d564cac41 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.176058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3076231-5959-4fc2-b204-ca5d564cac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.181298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee9609-69af-434a-a14c-05ed00187b52 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.181588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee9609-69af-434a-a14c-05ed00187b52 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.181866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee9609-69af-434a-a14c-05ed00187b52 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.183784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 94/188] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:34.188808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca71bd-b96b-44a7-87bc-4c9b2fb634e1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.189099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca71bd-b96b-44a7-87bc-4c9b2fb634e1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.189379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca71bd-b96b-44a7-87bc-4c9b2fb634e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.196604 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c3076231-5959-4fc2-b204-ca5d564cac41 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:34.197783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c3076231-5959-4fc2-b204-ca5d564cac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:34.207403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-54ca71bd-b96b-44a7-87bc-4c9b2fb634e1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:34.208590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-54ca71bd-b96b-44a7-87bc-4c9b2fb634e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:34.209567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:34.271158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 95/189] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:34.280293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 95/190] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:34.537692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3076231-5959-4fc2-b204-ca5d564cac41 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:34.551349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca71bd-b96b-44a7-87bc-4c9b2fb634e1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:34.630867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:34.646051 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3076231-5959-4fc2-b204-ca5d564cac41 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:34.657310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca71bd-b96b-44a7-87bc-4c9b2fb634e1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:34.682777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a8732-c2a4-4522-baec-2eea91674194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:34.683957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a8732-c2a4-4522-baec-2eea91674194 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.684213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a8732-c2a4-4522-baec-2eea91674194 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.684493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a8732-c2a4-4522-baec-2eea91674194 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.691818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bdc49-5c77-40a5-9b62-c75549cb9dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:34.692483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bdc49-5c77-40a5-9b62-c75549cb9dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.692767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bdc49-5c77-40a5-9b62-c75549cb9dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.693045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bdc49-5c77-40a5-9b62-c75549cb9dd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.694144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bdc49-5c77-40a5-9b62-c75549cb9dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:34.694853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a8732-c2a4-4522-baec-2eea91674194 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:34.695628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bdc49-5c77-40a5-9b62-c75549cb9dd8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:34.697209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 96/191] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-e81a8732-c2a4-4522-baec-2eea91674194 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:34.701128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:34.702303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:34.703522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 96/192] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:34.704199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bdc49-5c77-40a5-9b62-c75549cb9dd8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:34.708238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a8732-c2a4-4522-baec-2eea91674194 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:34.718170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:34.807813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bdc49-5c77-40a5-9b62-c75549cb9dd8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:34.809991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a8732-c2a4-4522-baec-2eea91674194 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:34.842199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec939b6-1603-43f7-b179-4d9704624b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:34.842799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec939b6-1603-43f7-b179-4d9704624b97 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.843109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec939b6-1603-43f7-b179-4d9704624b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.843383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec939b6-1603-43f7-b179-4d9704624b97 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.844025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec939b6-1603-43f7-b179-4d9704624b97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:34.844376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97c7c8-b704-4211-9415-6ece7c57f075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:34.844984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c7c8-b704-4211-9415-6ece7c57f075 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.845265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c7c8-b704-4211-9415-6ece7c57f075 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.845465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec939b6-1603-43f7-b179-4d9704624b97 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:34.845563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c7c8-b704-4211-9415-6ece7c57f075 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.846184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c7c8-b704-4211-9415-6ece7c57f075 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:34.846489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c7c8-b704-4211-9415-6ece7c57f075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:34.847877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c7c8-b704-4211-9415-6ece7c57f075 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:34.856083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 97/193] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:34.857278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfc9aa-1de0-48fb-a6bf-2756d8396d20 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.857565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfc9aa-1de0-48fb-a6bf-2756d8396d20 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.857853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dfc9aa-1de0-48fb-a6bf-2756d8396d20 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.859079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 98/194] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:34.863876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfc9aa-1de0-48fb-a6bf-2756d8396d20 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:34.871813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 97/195] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:34 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:34.873029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394014e0-8695-434d-87bb-7254d792b187 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:34.873316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394014e0-8695-434d-87bb-7254d792b187 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:34.873595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394014e0-8695-434d-87bb-7254d792b187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:34.880055 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:34.897007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-394014e0-8695-434d-87bb-7254d792b187 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:34.898240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-394014e0-8695-434d-87bb-7254d792b187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:34.953244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:34.978708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 98/196] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:34.997232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dfc9aa-1de0-48fb-a6bf-2756d8396d20 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:35.034982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace213e-421c-48cd-bf06-5325e23fcba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:35.035656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace213e-421c-48cd-bf06-5325e23fcba6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:35.035942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace213e-421c-48cd-bf06-5325e23fcba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:35.036215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace213e-421c-48cd-bf06-5325e23fcba6 None admin] PATH_INFO: `/v3/users/22ee40627dda401b9fd701b9585f1999` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:35.051373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace213e-421c-48cd-bf06-5325e23fcba6 None admin] RBAC: Authorizing `identity:get_user(user_id=22ee40627dda401b9fd701b9585f1999)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:35.052840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace213e-421c-48cd-bf06-5325e23fcba6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:35.054754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 99/197] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:34 2022] GET /identity/v3/users/22ee40627dda401b9fd701b9585f1999 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:35.060962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394014e0-8695-434d-87bb-7254d792b187 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:35.170398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394014e0-8695-434d-87bb-7254d792b187 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:35.204965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:35.210654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a96349-d78a-47fe-b5e2-dd7b1d416d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:35.211517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96349-d78a-47fe-b5e2-dd7b1d416d50 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:35.211670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96349-d78a-47fe-b5e2-dd7b1d416d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:35.211938 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a96349-d78a-47fe-b5e2-dd7b1d416d50 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:35.213068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a96349-d78a-47fe-b5e2-dd7b1d416d50 None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:35.214550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a96349-d78a-47fe-b5e2-dd7b1d416d50 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:35.216252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 99/198] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:35 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:35.221936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace213e-421c-48cd-bf06-5325e23fcba6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:35.259872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a96349-d78a-47fe-b5e2-dd7b1d416d50 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:35.343346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace213e-421c-48cd-bf06-5325e23fcba6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:35.371282 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a96349-d78a-47fe-b5e2-dd7b1d416d50 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:35.376644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:35.387095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1a390-c36e-448c-baaf-a6fe5d4ca5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:35.387989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1a390-c36e-448c-baaf-a6fe5d4ca5c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:35.388474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1a390-c36e-448c-baaf-a6fe5d4ca5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:35.388893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1a390-c36e-448c-baaf-a6fe5d4ca5c6 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/users/22ee40627dda401b9fd701b9585f1999/roles/542d76fcc5884c82aa0a0375961f29c2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:35.391000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1a390-c36e-448c-baaf-a6fe5d4ca5c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=97d9afe79c47436899dad4fbd1cd7bfd, user_id=22ee40627dda401b9fd701b9585f1999, role_id=542d76fcc5884c82aa0a0375961f29c2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:35.392615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1a390-c36e-448c-baaf-a6fe5d4ca5c6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:35.411236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 100/199] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 12:59:35 2022] PUT /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/users/22ee40627dda401b9fd701b9585f1999/roles/542d76fcc5884c82aa0a0375961f29c2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:35.412984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea047-cf91-4175-8649-86432cbc8749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:35.413630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea047-cf91-4175-8649-86432cbc8749 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:35.413915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea047-cf91-4175-8649-86432cbc8749 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:35.414190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aea047-cf91-4175-8649-86432cbc8749 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:35.429572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aea047-cf91-4175-8649-86432cbc8749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a4aea047-cf91-4175-8649-86432cbc8749 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:35.435931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:35.436884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:35.437695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 100/200] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:35.443781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1a390-c36e-448c-baaf-a6fe5d4ca5c6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:35.559641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1a390-c36e-448c-baaf-a6fe5d4ca5c6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:35.600022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:35.600564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:35.600848 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:35.601126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:35.601810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] RBAC: Adding query filter params (name=member) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:35.602102 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:35.603521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:35.613544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 101/201] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:35.619183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea047-cf91-4175-8649-86432cbc8749 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:35.682232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:35.731957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aea047-cf91-4175-8649-86432cbc8749 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:35.773223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8727d-c9b0-48f6-be21-54016ddb7efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:35.773859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8727d-c9b0-48f6-be21-54016ddb7efc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:35.774144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8727d-c9b0-48f6-be21-54016ddb7efc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:35.774423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8727d-c9b0-48f6-be21-54016ddb7efc None admin] PATH_INFO: `/v3/users/a10bfa156dbb46d3863840e801b7b0fe` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:35.775642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8727d-c9b0-48f6-be21-54016ddb7efc None admin] RBAC: Authorizing `identity:get_user(user_id=a10bfa156dbb46d3863840e801b7b0fe)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:35.777687 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8727d-c9b0-48f6-be21-54016ddb7efc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:35.779587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 101/202] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:35 2022] GET /identity/v3/users/a10bfa156dbb46d3863840e801b7b0fe => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:35.784931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:35.837802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:35.895316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8dbbf5-cf99-4bdb-b45a-2536147fa0ed None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:35.931118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998b64-83aa-43f2-9aad-de107b7af4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:35.932165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998b64-83aa-43f2-9aad-de107b7af4de None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:35.932720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998b64-83aa-43f2-9aad-de107b7af4de None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:35.933710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c998b64-83aa-43f2-9aad-de107b7af4de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:36.016641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c998b64-83aa-43f2-9aad-de107b7af4de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:36.018455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c998b64-83aa-43f2-9aad-de107b7af4de None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:36.075547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:36.122807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 102/203] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:36.128122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8727d-c9b0-48f6-be21-54016ddb7efc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:36.234429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8727d-c9b0-48f6-be21-54016ddb7efc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:36.262410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:36.270792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960ec4b-ac90-474e-a46c-1f1bcd8dacf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:36.271915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960ec4b-ac90-474e-a46c-1f1bcd8dacf4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:36.272199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960ec4b-ac90-474e-a46c-1f1bcd8dacf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:36.272478 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960ec4b-ac90-474e-a46c-1f1bcd8dacf4 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:36.273622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960ec4b-ac90-474e-a46c-1f1bcd8dacf4 None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:36.275095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960ec4b-ac90-474e-a46c-1f1bcd8dacf4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:36.277076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 102/204] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:36 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:36.283005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998b64-83aa-43f2-9aad-de107b7af4de None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:36.389431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c998b64-83aa-43f2-9aad-de107b7af4de None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:36.426805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f2c90-4bb4-41c1-a643-031451661795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:36.427459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2c90-4bb4-41c1-a643-031451661795 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:36.427743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2c90-4bb4-41c1-a643-031451661795 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:36.428017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f2c90-4bb4-41c1-a643-031451661795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:36.428813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f2c90-4bb4-41c1-a643-031451661795 None admin] RBAC: Adding query filter params (role.id=54e93caca0fb4c59a01ae64a0ed3a610, scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd, user.id=a10bfa156dbb46d3863840e801b7b0fe) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:36.429131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f2c90-4bb4-41c1-a643-031451661795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:36.430575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f2c90-4bb4-41c1-a643-031451661795 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:36.437547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 103/205] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:36 2022] GET /identity/v3/role_assignments?user.id=a10bfa156dbb46d3863840e801b7b0fe&scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd&role.id=54e93caca0fb4c59a01ae64a0ed3a610 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:37.265216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6a136-5a0e-49ad-ab90-6216691f630a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.265717 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6a136-5a0e-49ad-ab90-6216691f630a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.265820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6a136-5a0e-49ad-ab90-6216691f630a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.267268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 103/206] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:37.272446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d221c-f774-4f05-87bd-5505fb98b841 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.272732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d221c-f774-4f05-87bd-5505fb98b841 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.273006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d221c-f774-4f05-87bd-5505fb98b841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.291147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7c1d221c-f774-4f05-87bd-5505fb98b841 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:37.292316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c1d221c-f774-4f05-87bd-5505fb98b841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:37.300100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:37.304795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e3eae-ecde-4398-b7f0-07e7f9af9e03 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.305081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e3eae-ecde-4398-b7f0-07e7f9af9e03 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.305362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e3eae-ecde-4398-b7f0-07e7f9af9e03 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.306720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 104/207] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:59:37.312402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a96534-5a00-4793-aaee-6ebf5902a326 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.312685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a96534-5a00-4793-aaee-6ebf5902a326 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.312960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a96534-5a00-4793-aaee-6ebf5902a326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.334436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b5a96534-5a00-4793-aaee-6ebf5902a326 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:37.335586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b5a96534-5a00-4793-aaee-6ebf5902a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:37.374428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 104/208] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:37.409667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 105/209] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:37.648553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d221c-f774-4f05-87bd-5505fb98b841 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:37.691639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a96534-5a00-4793-aaee-6ebf5902a326 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:37.754947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d221c-f774-4f05-87bd-5505fb98b841 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:37.791275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e3274-3be0-4a1e-9e2f-3cf06682b5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:37.791903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e3274-3be0-4a1e-9e2f-3cf06682b5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.792182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e3274-3be0-4a1e-9e2f-3cf06682b5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.792481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e3274-3be0-4a1e-9e2f-3cf06682b5d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.792805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a96534-5a00-4793-aaee-6ebf5902a326 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:37.802919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e3274-3be0-4a1e-9e2f-3cf06682b5d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-136e3274-3be0-4a1e-9e2f-3cf06682b5d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:37.807808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:37.808763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:37.809535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 105/210] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:37.815251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d611ca-8be6-4eac-a706-c3fd3aa73071 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.815541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d611ca-8be6-4eac-a706-c3fd3aa73071 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.815820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d611ca-8be6-4eac-a706-c3fd3aa73071 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.817061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 106/211] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:37.817807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d611ca-8be6-4eac-a706-c3fd3aa73071 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:37.820698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:37.821168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:37.833780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbe614-3554-4fa0-904e-f6a15d43cb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:37.834414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe614-3554-4fa0-904e-f6a15d43cb20 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.834699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe614-3554-4fa0-904e-f6a15d43cb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.834972 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbe614-3554-4fa0-904e-f6a15d43cb20 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.836108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbe614-3554-4fa0-904e-f6a15d43cb20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:37.837575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbe614-3554-4fa0-904e-f6a15d43cb20 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:37.839163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 106/212] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:37.839163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fa0a2-7ded-488c-8055-05d4978edceb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.841254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fa0a2-7ded-488c-8055-05d4978edceb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.841441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fa0a2-7ded-488c-8055-05d4978edceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.859977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5b1fa0a2-7ded-488c-8055-05d4978edceb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:37.861150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5b1fa0a2-7ded-488c-8055-05d4978edceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:37.924545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d611ca-8be6-4eac-a706-c3fd3aa73071 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:37.927880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 107/213] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:37.928686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fa0a2-7ded-488c-8055-05d4978edceb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:37.967741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:37.968357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:37.968640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:37.968914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:37.969981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] RBAC: Adding query filter params (name=member) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:37.970306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:37.972156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:37.981757 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 107/214] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:37.987090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:38.034064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fa0a2-7ded-488c-8055-05d4978edceb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:38.051711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:38.062077 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:38.081121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015307a-0a5a-474b-a9f4-94cd4a12dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:38.082274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015307a-0a5a-474b-a9f4-94cd4a12dee5 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:38.082559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015307a-0a5a-474b-a9f4-94cd4a12dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:38.082991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015307a-0a5a-474b-a9f4-94cd4a12dee5 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:38.083684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015307a-0a5a-474b-a9f4-94cd4a12dee5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:38.085132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015307a-0a5a-474b-a9f4-94cd4a12dee5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:38.095734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1195bb-132d-4f1f-b6f6-356ece1bc44a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:38.106799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 108/215] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:37 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:38.136067 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34313e8-6d82-4a03-b8d7-14a7eba3d881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:38.137529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34313e8-6d82-4a03-b8d7-14a7eba3d881 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:38.137809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34313e8-6d82-4a03-b8d7-14a7eba3d881 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:38.138078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34313e8-6d82-4a03-b8d7-14a7eba3d881 None admin] PATH_INFO: `/v3/users/c847a20cb370415da702bda31f3e0c6e` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:38.153785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34313e8-6d82-4a03-b8d7-14a7eba3d881 None admin] RBAC: Authorizing `identity:get_user(user_id=c847a20cb370415da702bda31f3e0c6e)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:38.155287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34313e8-6d82-4a03-b8d7-14a7eba3d881 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:38.157198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 108/216] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:37 2022] GET /identity/v3/users/c847a20cb370415da702bda31f3e0c6e => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:38.163252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015307a-0a5a-474b-a9f4-94cd4a12dee5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:38.209278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34313e8-6d82-4a03-b8d7-14a7eba3d881 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:38.280123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015307a-0a5a-474b-a9f4-94cd4a12dee5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:38.322798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:38.324831 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:38.325248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:38.325520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:38.326348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] RBAC: Adding query filter params (role.id=54e93caca0fb4c59a01ae64a0ed3a610, user.id=c847a20cb370415da702bda31f3e0c6e, scope.system=all) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:38.327133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:38.327255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34313e8-6d82-4a03-b8d7-14a7eba3d881 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:38.328192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:38.334098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 109/217] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Oct 24 12:59:38 2022] GET /identity/v3/role_assignments?user.id=c847a20cb370415da702bda31f3e0c6e&scope.system=all&role.id=54e93caca0fb4c59a01ae64a0ed3a610 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:38.354857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:38.371426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b238c4d-89b4-4069-8028-58327c8e35ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:38.372340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238c4d-89b4-4069-8028-58327c8e35ac None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:38.372784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238c4d-89b4-4069-8028-58327c8e35ac None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:38.373063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b238c4d-89b4-4069-8028-58327c8e35ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:38.387675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b238c4d-89b4-4069-8028-58327c8e35ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-3b238c4d-89b4-4069-8028-58327c8e35ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:38.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:38.393770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:38.394564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 109/218] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:38.400247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:38.457961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:38.522437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee1bf-2966-4ed0-8498-ef8fe96dd172 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:38.557551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:38.558792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:38.559098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:38.559385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:38.560052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:38.560355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:38.561740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:38.572035 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 110/219] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:38.578602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b238c4d-89b4-4069-8028-58327c8e35ac None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:38.687637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b238c4d-89b4-4069-8028-58327c8e35ac None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:38.722720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0390f-9562-45b5-bbcf-c7360fe07bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:38.723640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0390f-9562-45b5-bbcf-c7360fe07bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:38.724094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0390f-9562-45b5-bbcf-c7360fe07bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:38.724515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0390f-9562-45b5-bbcf-c7360fe07bc7 None admin] PATH_INFO: `/v3/users/22ee40627dda401b9fd701b9585f1999` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:38.725817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0390f-9562-45b5-bbcf-c7360fe07bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=22ee40627dda401b9fd701b9585f1999)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:38.727436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0390f-9562-45b5-bbcf-c7360fe07bc7 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:38.729544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 110/220] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:38 2022] GET /identity/v3/users/22ee40627dda401b9fd701b9585f1999 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:38.736440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:38.847003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed84d5-007b-47f6-a656-f1f44ec20743 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:38.854188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:38.885598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bc81e-3325-40aa-a2ca-f2dacd08619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:38.886304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc81e-3325-40aa-a2ca-f2dacd08619c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:38.886641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc81e-3325-40aa-a2ca-f2dacd08619c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:38.886974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bc81e-3325-40aa-a2ca-f2dacd08619c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:38.960188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bc81e-3325-40aa-a2ca-f2dacd08619c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:38.961661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bc81e-3325-40aa-a2ca-f2dacd08619c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:39.067586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 111/221] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:39.077949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0390f-9562-45b5-bbcf-c7360fe07bc7 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:39.130389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:39.187639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0390f-9562-45b5-bbcf-c7360fe07bc7 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:39.224867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe44e8-5984-4b76-a4eb-40dd8c74e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:39.225851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe44e8-5984-4b76-a4eb-40dd8c74e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:39.226324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe44e8-5984-4b76-a4eb-40dd8c74e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:39.226805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe44e8-5984-4b76-a4eb-40dd8c74e2a9 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:39.228128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe44e8-5984-4b76-a4eb-40dd8c74e2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:39.229768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe44e8-5984-4b76-a4eb-40dd8c74e2a9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:39.232569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 111/222] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:39 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:39.242887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bc81e-3325-40aa-a2ca-f2dacd08619c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:39.247703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:39.350858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bc81e-3325-40aa-a2ca-f2dacd08619c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:39.386168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b7683-a3bd-4f14-98b4-4facb56a5cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:39.386708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b7683-a3bd-4f14-98b4-4facb56a5cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:39.386998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b7683-a3bd-4f14-98b4-4facb56a5cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:39.387313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b7683-a3bd-4f14-98b4-4facb56a5cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:39.388051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b7683-a3bd-4f14-98b4-4facb56a5cf0 None admin] RBAC: Adding query filter params (role.id=542d76fcc5884c82aa0a0375961f29c2, scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd, user.id=22ee40627dda401b9fd701b9585f1999) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:39.388367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b7683-a3bd-4f14-98b4-4facb56a5cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:39.389807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608b7683-a3bd-4f14-98b4-4facb56a5cf0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:39.396055 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 112/223] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 12:59:39 2022] GET /identity/v3/role_assignments?user.id=22ee40627dda401b9fd701b9585f1999&scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd&role.id=542d76fcc5884c82aa0a0375961f29c2 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:40.482363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0373b89-281a-43b5-9a6b-07e908b50418 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:40.482883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0373b89-281a-43b5-9a6b-07e908b50418 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:40.482883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0373b89-281a-43b5-9a6b-07e908b50418 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:40.484323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 112/224] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:40.489661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03044b-6f85-4e7c-893e-f255522a36a4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:40.490070 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03044b-6f85-4e7c-893e-f255522a36a4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:40.490214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03044b-6f85-4e7c-893e-f255522a36a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:40.510757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5a03044b-6f85-4e7c-893e-f255522a36a4 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:40.512067 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5a03044b-6f85-4e7c-893e-f255522a36a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:40.574499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29768df8-1f01-4fd0-b54e-cefed8890ec5 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:40.574786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29768df8-1f01-4fd0-b54e-cefed8890ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:40.575065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29768df8-1f01-4fd0-b54e-cefed8890ec5 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:40.576409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 113/225] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:40.581226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:40.582618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ba76a-c697-4682-bf93-590379f6d1cd None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:40.583165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ba76a-c697-4682-bf93-590379f6d1cd None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:40.583642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ba76a-c697-4682-bf93-590379f6d1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:40.587386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 113/226] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:40.602572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-872ba76a-c697-4682-bf93-590379f6d1cd None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:40.603768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-872ba76a-c697-4682-bf93-590379f6d1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:40.630234 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c181e-4f9f-4891-9e81-0274242d2e19 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:40.630517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c181e-4f9f-4891-9e81-0274242d2e19 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:40.630789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c181e-4f9f-4891-9e81-0274242d2e19 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:40.632536 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 114/227] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:59:40.637592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acd558-0514-4529-85d7-779ecc1d1d93 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:40.637879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acd558-0514-4529-85d7-779ecc1d1d93 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:40.638162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acd558-0514-4529-85d7-779ecc1d1d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:40.655728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-03acd558-0514-4529-85d7-779ecc1d1d93 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:40.656891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-03acd558-0514-4529-85d7-779ecc1d1d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:40.678491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 114/228] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:40.727659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 115/229] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:40.857258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ba76a-c697-4682-bf93-590379f6d1cd None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:40.908120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:40.959040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acd558-0514-4529-85d7-779ecc1d1d93 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:40.961135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ba76a-c697-4682-bf93-590379f6d1cd None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.001739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d3ecb-8e85-45c6-89b7-771e39edd3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.002381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d3ecb-8e85-45c6-89b7-771e39edd3dd None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.002667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d3ecb-8e85-45c6-89b7-771e39edd3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.002943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d3ecb-8e85-45c6-89b7-771e39edd3dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.014130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d3ecb-8e85-45c6-89b7-771e39edd3dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-6a6d3ecb-8e85-45c6-89b7-771e39edd3dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:41.019010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:41.020071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 115/230] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:41.021071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d3ecb-8e85-45c6-89b7-771e39edd3dd None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:41.068687 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acd558-0514-4529-85d7-779ecc1d1d93 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.097667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:41.103851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3733aa-39c9-4343-a858-b2634504d835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.104571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3733aa-39c9-4343-a858-b2634504d835 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.106009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3733aa-39c9-4343-a858-b2634504d835 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.106305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3733aa-39c9-4343-a858-b2634504d835 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.107495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3733aa-39c9-4343-a858-b2634504d835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:41.108881 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3733aa-39c9-4343-a858-b2634504d835 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:41.110465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 116/231] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:41.111253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3733aa-39c9-4343-a858-b2634504d835 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:41.127392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d3ecb-8e85-45c6-89b7-771e39edd3dd None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.137835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:41.162556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468861e7-7e51-488c-a551-8cd0129502f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.163301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468861e7-7e51-488c-a551-8cd0129502f4 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.163580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468861e7-7e51-488c-a551-8cd0129502f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.163849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468861e7-7e51-488c-a551-8cd0129502f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.174363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468861e7-7e51-488c-a551-8cd0129502f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-468861e7-7e51-488c-a551-8cd0129502f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:41.179272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:41.180185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 116/232] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:41.184578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468861e7-7e51-488c-a551-8cd0129502f4 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:41.212147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3733aa-39c9-4343-a858-b2634504d835 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.247589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.248186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.248465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.248738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.249405 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:41.249708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:41.251095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:41.259946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 117/233] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:41.260730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:41.287667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468861e7-7e51-488c-a551-8cd0129502f4 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.320516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b01270-b6bf-434e-8179-13c214a07917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.321148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b01270-b6bf-434e-8179-13c214a07917 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.321425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b01270-b6bf-434e-8179-13c214a07917 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.321697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b01270-b6bf-434e-8179-13c214a07917 None admin] PATH_INFO: `/v3/groups` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.322314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b01270-b6bf-434e-8179-13c214a07917 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:41.322891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:41.324755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b01270-b6bf-434e-8179-13c214a07917 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:41.341244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 117/234] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Oct 24 12:59:41 2022] POST /identity/v3/groups => generated 229 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:41.342059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b01270-b6bf-434e-8179-13c214a07917 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:41.373647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff947ae-afbc-40b2-a8bc-8011adb570a3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.380859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:41.413934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.414565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.414854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.415167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.416365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:41.416688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:41.418827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:41.434009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 118/235] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:41.435856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:41.474794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b01270-b6bf-434e-8179-13c214a07917 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.512964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90c365-9aca-48f8-9e06-2e2e27e7e397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.513655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90c365-9aca-48f8-9e06-2e2e27e7e397 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.513889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90c365-9aca-48f8-9e06-2e2e27e7e397 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.514165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90c365-9aca-48f8-9e06-2e2e27e7e397 None admin] PATH_INFO: `/v3/users/b3a1c8e59ae24730a18ba1b024cefa26` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.529984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90c365-9aca-48f8-9e06-2e2e27e7e397 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a1c8e59ae24730a18ba1b024cefa26)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:41.532040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90c365-9aca-48f8-9e06-2e2e27e7e397 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:41.533642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 118/236] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:41 2022] GET /identity/v3/users/b3a1c8e59ae24730a18ba1b024cefa26 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:41.539820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90c365-9aca-48f8-9e06-2e2e27e7e397 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:41.552679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79a626-4dd8-46ac-9ac8-3af9da29ff6e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.587000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:41.600098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242c2fa-3f6f-45fc-be09-8185bb3a3c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.600739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242c2fa-3f6f-45fc-be09-8185bb3a3c25 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.601023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242c2fa-3f6f-45fc-be09-8185bb3a3c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.601299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5242c2fa-3f6f-45fc-be09-8185bb3a3c25 None admin] PATH_INFO: `/v3/users/c847a20cb370415da702bda31f3e0c6e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.602970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242c2fa-3f6f-45fc-be09-8185bb3a3c25 None admin] RBAC: Authorizing `identity:get_user(user_id=c847a20cb370415da702bda31f3e0c6e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:41.604596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5242c2fa-3f6f-45fc-be09-8185bb3a3c25 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:41.607311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 119/237] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:41 2022] GET /identity/v3/users/c847a20cb370415da702bda31f3e0c6e => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:41.612561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242c2fa-3f6f-45fc-be09-8185bb3a3c25 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:41.639999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:41.659945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90c365-9aca-48f8-9e06-2e2e27e7e397 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.700016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02649a6-756f-4fc3-954d-d15582051b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.700660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02649a6-756f-4fc3-954d-d15582051b52 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.700903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02649a6-756f-4fc3-954d-d15582051b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.701173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02649a6-756f-4fc3-954d-d15582051b52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.701894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02649a6-756f-4fc3-954d-d15582051b52 None admin] RBAC: Adding query filter params (role.id=542d76fcc5884c82aa0a0375961f29c2, user.id=b3a1c8e59ae24730a18ba1b024cefa26, scope.system=all) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:41.702202 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02649a6-756f-4fc3-954d-d15582051b52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:41.703674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02649a6-756f-4fc3-954d-d15582051b52 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:41.709070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 119/238] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Oct 24 12:59:41 2022] GET /identity/v3/role_assignments?user.id=b3a1c8e59ae24730a18ba1b024cefa26&scope.system=all&role.id=542d76fcc5884c82aa0a0375961f29c2 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:41.721937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5242c2fa-3f6f-45fc-be09-8185bb3a3c25 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:41.760385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cf61b-5880-439c-94aa-0f92c1818b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:41.761154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cf61b-5880-439c-94aa-0f92c1818b5e None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:41.761434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cf61b-5880-439c-94aa-0f92c1818b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:41.761748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cf61b-5880-439c-94aa-0f92c1818b5e None admin] PATH_INFO: `/v3/system/users/c847a20cb370415da702bda31f3e0c6e/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:41.763249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cf61b-5880-439c-94aa-0f92c1818b5e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c847a20cb370415da702bda31f3e0c6e, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:41.764669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cf61b-5880-439c-94aa-0f92c1818b5e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:41.783439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 120/239] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Oct 24 12:59:41 2022] PUT /identity/v3/system/users/c847a20cb370415da702bda31f3e0c6e/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 12:59:43.541683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d5ed-c5dd-45a3-be54-25f15eacf0ef None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:43.542191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d5ed-c5dd-45a3-be54-25f15eacf0ef None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:43.542745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552d5ed-c5dd-45a3-be54-25f15eacf0ef None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:43.544180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 120/240] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:43.549311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5be1c-9f06-4a5b-946e-eb4a52f28e95 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:43.549695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5be1c-9f06-4a5b-946e-eb4a52f28e95 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:43.549874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5be1c-9f06-4a5b-946e-eb4a52f28e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:43.568777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f6c5be1c-9f06-4a5b-946e-eb4a52f28e95 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:43.569965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f6c5be1c-9f06-4a5b-946e-eb4a52f28e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:43.622541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:43.641818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 121/241] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:43.872153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3350-e239-466d-9c3e-dc7c1908436b None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:43.872485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3350-e239-466d-9c3e-dc7c1908436b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:43.872660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cf3350-e239-466d-9c3e-dc7c1908436b None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:43.874106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 121/242] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:43.879016 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de56e9-faa2-4b98-ae9b-0347955bba01 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:43.879343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de56e9-faa2-4b98-ae9b-0347955bba01 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:43.879625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de56e9-faa2-4b98-ae9b-0347955bba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:43.898375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-41de56e9-faa2-4b98-ae9b-0347955bba01 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:43.899576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-41de56e9-faa2-4b98-ae9b-0347955bba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:43.914491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf3350-e239-466d-9c3e-dc7c1908436b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:43.969572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 122/243] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:43.969572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe529-1000-4a25-992e-58493586c323 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:43.969572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe529-1000-4a25-992e-58493586c323 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:43.971449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fe529-1000-4a25-992e-58493586c323 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:43.972754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 123/244] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:43.977933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb368b-d99f-45f2-96de-e53e5cf20f7c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:43.978216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb368b-d99f-45f2-96de-e53e5cf20f7c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:43.978492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb368b-d99f-45f2-96de-e53e5cf20f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:43.997691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0abb368b-d99f-45f2-96de-e53e5cf20f7c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:43.998859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0abb368b-d99f-45f2-96de-e53e5cf20f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:44.024225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cf3350-e239-466d-9c3e-dc7c1908436b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.059851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866d110-000d-4337-be77-ef0a5dd322bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.060488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866d110-000d-4337-be77-ef0a5dd322bf None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.060773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866d110-000d-4337-be77-ef0a5dd322bf None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.061047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866d110-000d-4337-be77-ef0a5dd322bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.063168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866d110-000d-4337-be77-ef0a5dd322bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:44.064661 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866d110-000d-4337-be77-ef0a5dd322bf None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:44.065377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 122/245] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:44.070536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866d110-000d-4337-be77-ef0a5dd322bf None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.071034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 124/246] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:44.092038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:44.175635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866d110-000d-4337-be77-ef0a5dd322bf None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.209889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2fc86-08bd-4d28-826b-a388f25e819e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.210497 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2fc86-08bd-4d28-826b-a388f25e819e None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.210778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2fc86-08bd-4d28-826b-a388f25e819e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.211133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2fc86-08bd-4d28-826b-a388f25e819e None admin] PATH_INFO: `/v3/groups` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.211760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2fc86-08bd-4d28-826b-a388f25e819e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:44.213255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2fc86-08bd-4d28-826b-a388f25e819e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:44.229566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 123/247] 10.222.0.42 () {60 vars in 1134 bytes} [Mon Oct 24 12:59:44 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 12:59:44.241796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb368b-d99f-45f2-96de-e53e5cf20f7c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.296400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:44.351826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2fc86-08bd-4d28-826b-a388f25e819e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.365053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb368b-d99f-45f2-96de-e53e5cf20f7c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.404662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a6899-081c-43a5-910e-cc9770e44d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.405309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6899-081c-43a5-910e-cc9770e44d53 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.405596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6899-081c-43a5-910e-cc9770e44d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.405876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a6899-081c-43a5-910e-cc9770e44d53 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.416861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a6899-081c-43a5-910e-cc9770e44d53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ab7a6899-081c-43a5-910e-cc9770e44d53 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:44.421850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:44.422762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:44.423786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 125/248] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:44.428811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a6899-081c-43a5-910e-cc9770e44d53 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.449938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:44.474087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2fc86-08bd-4d28-826b-a388f25e819e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.514397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92816c6b-1e59-45bb-8952-d541f3011e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.515970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92816c6b-1e59-45bb-8952-d541f3011e02 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.516468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92816c6b-1e59-45bb-8952-d541f3011e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.516947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92816c6b-1e59-45bb-8952-d541f3011e02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.535151 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92816c6b-1e59-45bb-8952-d541f3011e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-92816c6b-1e59-45bb-8952-d541f3011e02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:44.541543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:44.543758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:44.544739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 124/249] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:44.549414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92816c6b-1e59-45bb-8952-d541f3011e02 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.562030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a6899-081c-43a5-910e-cc9770e44d53 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.566741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:44.606312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.606940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.607412 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.607697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.608383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:44.608813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:44.610215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:44.619636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 126/250] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:44.627241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.678422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92816c6b-1e59-45bb-8952-d541f3011e02 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.720626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.721174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.721460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.721743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.722413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] RBAC: Adding query filter params (name=member) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:44.723007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:44.724701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:44.736373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:44.739527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 125/251] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:44.742722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.768452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8335d10-372d-42b0-9d6a-9a0ae28e9f79 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.816244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25bdd2-e18d-4d90-908b-090998380f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.816886 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25bdd2-e18d-4d90-908b-090998380f57 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.817167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25bdd2-e18d-4d90-908b-090998380f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.817445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25bdd2-e18d-4d90-908b-090998380f57 None admin] PATH_INFO: `/v3/users/b3a1c8e59ae24730a18ba1b024cefa26` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.818583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25bdd2-e18d-4d90-908b-090998380f57 None admin] RBAC: Authorizing `identity:get_user(user_id=b3a1c8e59ae24730a18ba1b024cefa26)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:44.820139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25bdd2-e18d-4d90-908b-090998380f57 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:44.821738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 127/252] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:44 2022] GET /identity/v3/users/b3a1c8e59ae24730a18ba1b024cefa26 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:44.827937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25bdd2-e18d-4d90-908b-090998380f57 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.864426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:44.866558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4aa02-c5d9-443a-8b74-d016b178a12e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.903989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b25f9-af8e-4ab7-be65-8ec23bcf324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.904425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b25f9-af8e-4ab7-be65-8ec23bcf324f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.904715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b25f9-af8e-4ab7-be65-8ec23bcf324f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.905111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b25f9-af8e-4ab7-be65-8ec23bcf324f None admin] PATH_INFO: `/v3/users/c847a20cb370415da702bda31f3e0c6e` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.906158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b25f9-af8e-4ab7-be65-8ec23bcf324f None admin] RBAC: Authorizing `identity:get_user(user_id=c847a20cb370415da702bda31f3e0c6e)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:44.908616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b25f9-af8e-4ab7-be65-8ec23bcf324f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:44.909991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 126/253] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:44 2022] GET /identity/v3/users/c847a20cb370415da702bda31f3e0c6e => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:44.925258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b25f9-af8e-4ab7-be65-8ec23bcf324f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:44.944676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25bdd2-e18d-4d90-908b-090998380f57 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:44.983031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf5455-8102-43c8-942f-f0e8a4c1520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:44.983939 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf5455-8102-43c8-942f-f0e8a4c1520c None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:44.984368 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf5455-8102-43c8-942f-f0e8a4c1520c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:44.984791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf5455-8102-43c8-942f-f0e8a4c1520c None admin] PATH_INFO: `/v3/system/users/b3a1c8e59ae24730a18ba1b024cefa26/roles/542d76fcc5884c82aa0a0375961f29c2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:44.986397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf5455-8102-43c8-942f-f0e8a4c1520c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b3a1c8e59ae24730a18ba1b024cefa26, role_id=542d76fcc5884c82aa0a0375961f29c2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:44.988476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf5455-8102-43c8-942f-f0e8a4c1520c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:45.006592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 128/254] 10.222.0.42 () {58 vars in 1334 bytes} [Mon Oct 24 12:59:44 2022] PUT /identity/v3/system/users/b3a1c8e59ae24730a18ba1b024cefa26/roles/542d76fcc5884c82aa0a0375961f29c2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 12:59:45.016056 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:45.074391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b25f9-af8e-4ab7-be65-8ec23bcf324f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:45.113509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d84d4-f64f-48ff-a303-65f7ce1e1400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:45.114144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d84d4-f64f-48ff-a303-65f7ce1e1400 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:45.114428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d84d4-f64f-48ff-a303-65f7ce1e1400 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:45.114704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d84d4-f64f-48ff-a303-65f7ce1e1400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:45.115492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d84d4-f64f-48ff-a303-65f7ce1e1400 None admin] RBAC: Adding query filter params (role.id=54e93caca0fb4c59a01ae64a0ed3a610, user.id=c847a20cb370415da702bda31f3e0c6e, scope.system=all) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:45.115810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d84d4-f64f-48ff-a303-65f7ce1e1400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:45.117364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d84d4-f64f-48ff-a303-65f7ce1e1400 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:45.132501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 127/255] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Oct 24 12:59:44 2022] GET /identity/v3/role_assignments?user.id=c847a20cb370415da702bda31f3e0c6e&scope.system=all&role.id=54e93caca0fb4c59a01ae64a0ed3a610 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:46.934538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde388d-2782-4827-b9ea-e58ea3130cd1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:46.935163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde388d-2782-4827-b9ea-e58ea3130cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:46.935163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde388d-2782-4827-b9ea-e58ea3130cd1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:46.936607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 129/256] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:46.942079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb5f67-9a91-4954-86f8-7b4df18d6916 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:46.942490 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb5f67-9a91-4954-86f8-7b4df18d6916 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:46.942656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb5f67-9a91-4954-86f8-7b4df18d6916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:46.963782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-73bb5f67-9a91-4954-86f8-7b4df18d6916 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:46.965528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-73bb5f67-9a91-4954-86f8-7b4df18d6916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:47.042952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 128/257] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 12:59:47.282732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65f4e-2117-4af7-8ac8-451f1f38d026 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:47.283105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65f4e-2117-4af7-8ac8-451f1f38d026 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:47.283283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe65f4e-2117-4af7-8ac8-451f1f38d026 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:47.284753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 130/258] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:47.290399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d3e5b-1b84-44cf-bae8-1c7bf925b934 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:47.290696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d3e5b-1b84-44cf-bae8-1c7bf925b934 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:47.290978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d3e5b-1b84-44cf-bae8-1c7bf925b934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:47.312018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-863d3e5b-1b84-44cf-bae8-1c7bf925b934 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:47.313228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-863d3e5b-1b84-44cf-bae8-1c7bf925b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:47.327277 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 129/259] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 12:59:47.615429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe65f4e-2117-4af7-8ac8-451f1f38d026 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:47.633589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe65f4e-2117-4af7-8ac8-451f1f38d026 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:47.640950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb784c1f-b36b-4896-8733-b8357236a522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:47.641594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb784c1f-b36b-4896-8733-b8357236a522 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:47.641882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb784c1f-b36b-4896-8733-b8357236a522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:47.642156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb784c1f-b36b-4896-8733-b8357236a522 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:47.654375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb784c1f-b36b-4896-8733-b8357236a522 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-eb784c1f-b36b-4896-8733-b8357236a522 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:47.659286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:47.660144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:47.660944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:47.665199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 131/260] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 24 12:59:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:47.666572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d3e5b-1b84-44cf-bae8-1c7bf925b934 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:47.677923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d3e5b-1b84-44cf-bae8-1c7bf925b934 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:47.691774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:47.692417 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:47.692706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:47.692989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:47.693677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:47.693991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:47.695841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING py.warnings [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 12:59:47.695841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: warnings.warn(msg) Oct 24 12:59:47.695841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: Oct 24 12:59:47.696796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:47.709282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 130/261] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 12:59:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:47.716850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb784c1f-b36b-4896-8733-b8357236a522 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:47.731020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb784c1f-b36b-4896-8733-b8357236a522 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:47.740626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:47.741283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:47.741565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:47.741879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] PATH_INFO: `/v3/groups/46b7dac9dfb84e58baf059cba923fbeb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:47.751657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] RBAC: Authorizing `identity:get_group(group_id=46b7dac9dfb84e58baf059cba923fbeb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:47.753623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 12:59:47.753623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 12:59:47.753623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 12:59:47.754877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:47.756913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 132/262] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 24 12:59:47 2022] GET /identity/v3/groups/46b7dac9dfb84e58baf059cba923fbeb => generated 226 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:47.764577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:47.781676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c85d5-2e2d-48ae-91fa-d20144205276 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:47.790334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d12b16-9a94-45c5-8884-1861163d0a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:47.790990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d12b16-9a94-45c5-8884-1861163d0a79 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:47.791303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d12b16-9a94-45c5-8884-1861163d0a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:47.791584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d12b16-9a94-45c5-8884-1861163d0a79 admin admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:47.795923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d12b16-9a94-45c5-8884-1861163d0a79 admin admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:47.798058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d12b16-9a94-45c5-8884-1861163d0a79 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:47.799826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 131/263] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 24 12:59:47 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:47.806381 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:47.821323 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110377f7-f477-4774-95f5-c8f427d175e7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:47.830223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:47.830914 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:47.831230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:47.831511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2/groups/46b7dac9dfb84e58baf059cba923fbeb/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:47.836354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1634123c90c5493296570578db87c5c2, group_id=46b7dac9dfb84e58baf059cba923fbeb, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:47.837394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 12:59:47.837394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 12:59:47.837394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 12:59:47.840011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:47.865582 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 133/264] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 24 12:59:47 2022] PUT /identity/v3/projects/1634123c90c5493296570578db87c5c2/groups/46b7dac9dfb84e58baf059cba923fbeb/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:47.889078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d12b16-9a94-45c5-8884-1861163d0a79 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:47.903483 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d12b16-9a94-45c5-8884-1861163d0a79 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:47.976783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:47.985156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f20691-d650-420e-abbf-f9cfb22ddfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:47.985818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f20691-d650-420e-abbf-f9cfb22ddfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:47.986105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f20691-d650-420e-abbf-f9cfb22ddfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:47.986381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f20691-d650-420e-abbf-f9cfb22ddfc0 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:47.999403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f20691-d650-420e-abbf-f9cfb22ddfc0 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-38f20691-d650-420e-abbf-f9cfb22ddfc0 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:48.006928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:48.009333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:48.011915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 132/265] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 24 12:59:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 120 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:48.019342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.034840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f3319b-4f23-416d-891e-428bb3e536ec admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.046204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.046839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.047162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.047447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.048318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:48.048635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:48.049649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 12:59:48.049649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 12:59:48.049649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 12:59:48.050849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:48.066419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c82d07-6136-44c9-8c20-9dfdccdcdc83 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.066705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 134/266] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:48.066855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c82d07-6136-44c9-8c20-9dfdccdcdc83 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.066855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c82d07-6136-44c9-8c20-9dfdccdcdc83 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.066855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 133/267] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:48.079293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.080875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f32f28-9c6c-418b-961c-eda10238b704 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.081166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f32f28-9c6c-418b-961c-eda10238b704 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.081446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f32f28-9c6c-418b-961c-eda10238b704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.093475 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca156e4-548b-4a1b-85a0-8e54a6b89af3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.103042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2a46-2db5-43b7-ad44-cb72a6aea05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.103723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2a46-2db5-43b7-ad44-cb72a6aea05f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.104008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2a46-2db5-43b7-ad44-cb72a6aea05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.104290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2a46-2db5-43b7-ad44-cb72a6aea05f admin admin] PATH_INFO: `/v3/groups/46b7dac9dfb84e58baf059cba923fbeb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.109808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af2a46-2db5-43b7-ad44-cb72a6aea05f admin admin] RBAC: Authorizing `identity:get_group(group_id=46b7dac9dfb84e58baf059cba923fbeb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:48.111826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af2a46-2db5-43b7-ad44-cb72a6aea05f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:48.113503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 135/268] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/groups/46b7dac9dfb84e58baf059cba923fbeb => generated 226 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:48.119198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2a46-2db5-43b7-ad44-cb72a6aea05f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.121946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d8f32f28-9c6c-418b-961c-eda10238b704 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:48.123254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d8f32f28-9c6c-418b-961c-eda10238b704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:48.134415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2a46-2db5-43b7-ad44-cb72a6aea05f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.134945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 134/269] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 12:59:48.136648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999e88d-056c-40fe-b50a-9228a0765bae None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.137033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999e88d-056c-40fe-b50a-9228a0765bae None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.137299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999e88d-056c-40fe-b50a-9228a0765bae None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.138822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 135/270] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:48.144299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd781f0-e488-4fc4-ade3-ba1f8c558a3f None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.144651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd781f0-e488-4fc4-ade3-ba1f8c558a3f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.144993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd781f0-e488-4fc4-ade3-ba1f8c558a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.147705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8add37-43cf-453b-99a2-3ce72803c654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.148436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8add37-43cf-453b-99a2-3ce72803c654 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.148733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8add37-43cf-453b-99a2-3ce72803c654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.149116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8add37-43cf-453b-99a2-3ce72803c654 admin admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.150390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8add37-43cf-453b-99a2-3ce72803c654 admin admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:48.154160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8add37-43cf-453b-99a2-3ce72803c654 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:48.155753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 136/271] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:48.162245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8add37-43cf-453b-99a2-3ce72803c654 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.166908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9cd781f0-e488-4fc4-ade3-ba1f8c558a3f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:48.168239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9cd781f0-e488-4fc4-ade3-ba1f8c558a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:48.183644 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8add37-43cf-453b-99a2-3ce72803c654 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.197227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca01ec-ff48-4b57-b171-6def63860ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.198022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca01ec-ff48-4b57-b171-6def63860ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.199022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca01ec-ff48-4b57-b171-6def63860ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.199394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca01ec-ff48-4b57-b171-6def63860ec2 admin admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2/groups/46b7dac9dfb84e58baf059cba923fbeb/roles/78ff1dcdc39743048089711fcfc4d3a4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.204792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca01ec-ff48-4b57-b171-6def63860ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1634123c90c5493296570578db87c5c2, group_id=46b7dac9dfb84e58baf059cba923fbeb, role_id=78ff1dcdc39743048089711fcfc4d3a4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:48.207878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca01ec-ff48-4b57-b171-6def63860ec2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:48.243723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 137/272] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 24 12:59:48 2022] PUT /identity/v3/projects/1634123c90c5493296570578db87c5c2/groups/46b7dac9dfb84e58baf059cba923fbeb/roles/78ff1dcdc39743048089711fcfc4d3a4 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 12:59:48.257268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 136/273] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:48.570330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca01ec-ff48-4b57-b171-6def63860ec2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.591705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd781f0-e488-4fc4-ade3-ba1f8c558a3f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.616818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:48.692146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd781f0-e488-4fc4-ade3-ba1f8c558a3f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.699078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad0923-0f8b-45c6-9d47-92f4024af1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.700211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad0923-0f8b-45c6-9d47-92f4024af1f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.700677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad0923-0f8b-45c6-9d47-92f4024af1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.701103 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad0923-0f8b-45c6-9d47-92f4024af1f2 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.715289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca01ec-ff48-4b57-b171-6def63860ec2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.719917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fad0923-0f8b-45c6-9d47-92f4024af1f2 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-3fad0923-0f8b-45c6-9d47-92f4024af1f2 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:48.725477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:48.726348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:48.727210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:48.727802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 137/274] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 136 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:48.728587 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac185d-1413-4cc9-af61-0092f6034517 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.728881 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac185d-1413-4cc9-af61-0092f6034517 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.729160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac185d-1413-4cc9-af61-0092f6034517 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.730728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 138/275] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:48.734035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac185d-1413-4cc9-af61-0092f6034517 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.759944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac185d-1413-4cc9-af61-0092f6034517 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.769823 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f5478-9cf5-4851-badb-33386d79d2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.771211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5478-9cf5-4851-badb-33386d79d2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.771525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5478-9cf5-4851-badb-33386d79d2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.771904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f5478-9cf5-4851-badb-33386d79d2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.772638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f5478-9cf5-4851-badb-33386d79d2a3 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:48.772955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f5478-9cf5-4851-badb-33386d79d2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:48.774844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60f5478-9cf5-4851-badb-33386d79d2a3 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:48.781154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757c817-0bf7-4b3a-9d94-8970b9426555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.782072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757c817-0bf7-4b3a-9d94-8970b9426555 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.782554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757c817-0bf7-4b3a-9d94-8970b9426555 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.783026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757c817-0bf7-4b3a-9d94-8970b9426555 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.785575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 139/276] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:48.787035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb93ad-02ac-4691-8d44-de91b592a5ff None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.787348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb93ad-02ac-4691-8d44-de91b592a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.787625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb93ad-02ac-4691-8d44-de91b592a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.810209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-b6cb93ad-02ac-4691-8d44-de91b592a5ff None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:48.811995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b6cb93ad-02ac-4691-8d44-de91b592a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:48.819994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757c817-0bf7-4b3a-9d94-8970b9426555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:48.820795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 140/277] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 24 12:59:48.821625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb93ad-02ac-4691-8d44-de91b592a5ff None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f757c817-0bf7-4b3a-9d94-8970b9426555 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:48.826868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:48.827822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 24 12:59:48.828614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:48.831184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 138/278] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:48.841075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb93ad-02ac-4691-8d44-de91b592a5ff None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.850480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.851211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.851496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.851833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] PATH_INFO: `/v3/groups/46b7dac9dfb84e58baf059cba923fbeb` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.852425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757c817-0bf7-4b3a-9d94-8970b9426555 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.855796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] RBAC: Authorizing `identity:get_group(group_id=46b7dac9dfb84e58baf059cba923fbeb)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:48.859076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING py.warnings [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 12:59:48.859076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: warnings.warn(msg) Oct 24 12:59:48.859076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: Oct 24 12:59:48.859076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:48.861779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 141/279] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/groups/46b7dac9dfb84e58baf059cba923fbeb => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:48.871077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.889555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30573f8-43f8-4ec1-868a-f2c1c70b53b4 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.900807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8eda4-58ab-4341-b424-d4a613817784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.902007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8eda4-58ab-4341-b424-d4a613817784 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.902301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8eda4-58ab-4341-b424-d4a613817784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.902580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d8eda4-58ab-4341-b424-d4a613817784 admin admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.903974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d8eda4-58ab-4341-b424-d4a613817784 admin admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:48.906153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d8eda4-58ab-4341-b424-d4a613817784 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:48.907852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 142/280] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:48.914332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8eda4-58ab-4341-b424-d4a613817784 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:48.931320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d8eda4-58ab-4341-b424-d4a613817784 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:48.947764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:48.948489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:48.948786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:48.949070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/groups/46b7dac9dfb84e58baf059cba923fbeb/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:48.951271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d9afe79c47436899dad4fbd1cd7bfd, group_id=46b7dac9dfb84e58baf059cba923fbeb, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:48.952309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING py.warnings [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 12:59:48.952309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: warnings.warn(msg) Oct 24 12:59:48.952309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: Oct 24 12:59:48.954271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:48.980835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 143/281] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 24 12:59:48 2022] PUT /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/groups/46b7dac9dfb84e58baf059cba923fbeb/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:49.041610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:49.049149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757c817-0bf7-4b3a-9d94-8970b9426555 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:49.114792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:49.115797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.116280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.116758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.117647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:49.118149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:49.119735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:49.135201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 139/282] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:49.141714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:49.179214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:49.277937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21183b-409e-457f-82ed-adc6e1da5577 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:49.309067 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:49.319434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d9a0-8707-48c1-b9e4-45f8c2faaf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:49.320087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d9a0-8707-48c1-b9e4-45f8c2faaf1c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.320373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d9a0-8707-48c1-b9e4-45f8c2faaf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.320656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8d9a0-8707-48c1-b9e4-45f8c2faaf1c None admin] PATH_INFO: `/v3/users/b3a1c8e59ae24730a18ba1b024cefa26` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.321839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8d9a0-8707-48c1-b9e4-45f8c2faaf1c None admin] RBAC: Authorizing `identity:get_user(user_id=b3a1c8e59ae24730a18ba1b024cefa26)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:49.323933 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8d9a0-8707-48c1-b9e4-45f8c2faaf1c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:49.325726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 144/283] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 12:59:49 2022] GET /identity/v3/users/b3a1c8e59ae24730a18ba1b024cefa26 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:49.331831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d9a0-8707-48c1-b9e4-45f8c2faaf1c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:49.482774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd08139-52eb-41fa-a1db-c3c5210fa9b7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:49.490690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe0115-4ef3-4209-b2b9-d48a5ed1c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:49.491725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe0115-4ef3-4209-b2b9-d48a5ed1c57b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.492017 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe0115-4ef3-4209-b2b9-d48a5ed1c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.492292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe0115-4ef3-4209-b2b9-d48a5ed1c57b admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.497364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:49.516469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbe0115-4ef3-4209-b2b9-d48a5ed1c57b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:49.522257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8d9a0-8707-48c1-b9e4-45f8c2faaf1c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-dbbe0115-4ef3-4209-b2b9-d48a5ed1c57b admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:49.522786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:49.523685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:49.524461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:49.532781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 140/284] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 24 12:59:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:49.538751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe0115-4ef3-4209-b2b9-d48a5ed1c57b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:49.553025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe0115-4ef3-4209-b2b9-d48a5ed1c57b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:49.558666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:49.559759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.560049 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.560322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.561006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:49.561312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:49.565592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:49.578537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 141/285] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 24 12:59:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:49.584155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:49.600149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe428c-67aa-4e44-bc37-21c931bf6dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:49.600604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b3a80-0290-4cf5-be6b-b43622cdb07d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:49.600822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe428c-67aa-4e44-bc37-21c931bf6dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.601104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe428c-67aa-4e44-bc37-21c931bf6dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.601426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe428c-67aa-4e44-bc37-21c931bf6dd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.602166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe428c-67aa-4e44-bc37-21c931bf6dd8 None admin] RBAC: Adding query filter params (role.id=542d76fcc5884c82aa0a0375961f29c2, user.id=b3a1c8e59ae24730a18ba1b024cefa26, scope.system=all) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:49.602578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe428c-67aa-4e44-bc37-21c931bf6dd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:49.607329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe428c-67aa-4e44-bc37-21c931bf6dd8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:49.609045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da251541-4a64-46d4-8aa2-889a89c1eba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:49.609999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251541-4a64-46d4-8aa2-889a89c1eba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.610894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251541-4a64-46d4-8aa2-889a89c1eba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.611212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da251541-4a64-46d4-8aa2-889a89c1eba8 admin admin] PATH_INFO: `/v3/groups/46b7dac9dfb84e58baf059cba923fbeb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.612572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da251541-4a64-46d4-8aa2-889a89c1eba8 admin admin] RBAC: Authorizing `identity:get_group(group_id=46b7dac9dfb84e58baf059cba923fbeb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:49.615225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da251541-4a64-46d4-8aa2-889a89c1eba8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:49.621794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 142/286] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 24 12:59:49 2022] GET /identity/v3/groups/46b7dac9dfb84e58baf059cba923fbeb => generated 226 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:49.622731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da251541-4a64-46d4-8aa2-889a89c1eba8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:49.626091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 145/287] 10.222.0.42 () {56 vars in 1308 bytes} [Mon Oct 24 12:59:49 2022] GET /identity/v3/role_assignments?user.id=b3a1c8e59ae24730a18ba1b024cefa26&scope.system=all&role.id=542d76fcc5884c82aa0a0375961f29c2 => generated 510 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:49.630908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd96a55-af6e-4f98-8a1a-e98c93bfb26f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.631474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd96a55-af6e-4f98-8a1a-e98c93bfb26f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.631753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd96a55-af6e-4f98-8a1a-e98c93bfb26f None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.633393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 146/288] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:59:49.637686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da251541-4a64-46d4-8aa2-889a89c1eba8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:49.639311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a95a5-1de5-436b-bd9d-3a33c5bec4db None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.639776 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a95a5-1de5-436b-bd9d-3a33c5bec4db None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.640055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a95a5-1de5-436b-bd9d-3a33c5bec4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.647403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45bfe7-7673-4242-8e54-26973da38f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:49.648076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bfe7-7673-4242-8e54-26973da38f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.648359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bfe7-7673-4242-8e54-26973da38f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.648745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45bfe7-7673-4242-8e54-26973da38f0a admin admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.650788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45bfe7-7673-4242-8e54-26973da38f0a admin admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:49.653484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45bfe7-7673-4242-8e54-26973da38f0a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:49.659277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 143/289] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 24 12:59:49 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:49.661676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45bfe7-7673-4242-8e54-26973da38f0a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:49.675981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45bfe7-7673-4242-8e54-26973da38f0a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:49.677864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9c3a95a5-1de5-436b-bd9d-3a33c5bec4db None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:49.679356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9c3a95a5-1de5-436b-bd9d-3a33c5bec4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:49.683207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65e47-90cb-4fff-8174-c46fafd87224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:49.684335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65e47-90cb-4fff-8174-c46fafd87224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:49.684760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65e47-90cb-4fff-8174-c46fafd87224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:49.685158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf65e47-90cb-4fff-8174-c46fafd87224 admin admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/groups/46b7dac9dfb84e58baf059cba923fbeb/roles/78ff1dcdc39743048089711fcfc4d3a4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:49.687830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf65e47-90cb-4fff-8174-c46fafd87224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d9afe79c47436899dad4fbd1cd7bfd, group_id=46b7dac9dfb84e58baf059cba923fbeb, role_id=78ff1dcdc39743048089711fcfc4d3a4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:49.690796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf65e47-90cb-4fff-8174-c46fafd87224 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:49.692512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 147/290] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 24 12:59:49.707926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 144/291] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 24 12:59:49 2022] PUT /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd/groups/46b7dac9dfb84e58baf059cba923fbeb/roles/78ff1dcdc39743048089711fcfc4d3a4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 12:59:50.016077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a95a5-1de5-436b-bd9d-3a33c5bec4db None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:50.026105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a95a5-1de5-436b-bd9d-3a33c5bec4db None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:50.094847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfaf4d-3315-4676-af33-20a56223f297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:50.095514 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfaf4d-3315-4676-af33-20a56223f297 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:50.095794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfaf4d-3315-4676-af33-20a56223f297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:50.096069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfaf4d-3315-4676-af33-20a56223f297 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:50.109850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfaf4d-3315-4676-af33-20a56223f297 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-1edfaf4d-3315-4676-af33-20a56223f297 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:50.115014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:50.117125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:50.118551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 148/292] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 24 12:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 101 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:50.122601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65e47-90cb-4fff-8174-c46fafd87224 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:50.133161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf65e47-90cb-4fff-8174-c46fafd87224 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:50.138154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:50.138773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:50.139043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:50.139346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:50.140019 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:50.140317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:50.141985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:50.152300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 145/293] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 24 12:59:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:50.157654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfaf4d-3315-4676-af33-20a56223f297 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:50.168479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfaf4d-3315-4676-af33-20a56223f297 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:50.174756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bc734-1b64-4948-8882-e5f163d78bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:50.175436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bc734-1b64-4948-8882-e5f163d78bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:50.175717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bc734-1b64-4948-8882-e5f163d78bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:50.175987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8bc734-1b64-4948-8882-e5f163d78bc9 admin admin] PATH_INFO: `/v3/groups/1cf2dbe75dfa4b3ca970768c8f1a52a8` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:50.184656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8bc734-1b64-4948-8882-e5f163d78bc9 admin admin] RBAC: Authorizing `identity:get_group(group_id=1cf2dbe75dfa4b3ca970768c8f1a52a8)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:50.186660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8bc734-1b64-4948-8882-e5f163d78bc9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:50.188682 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 149/294] 10.222.0.42 () {56 vars in 1201 bytes} [Mon Oct 24 12:59:50 2022] GET /identity/v3/groups/1cf2dbe75dfa4b3ca970768c8f1a52a8 => generated 229 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:50.193818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:50.207656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04717754-7685-40be-a5cd-e44fd8b36009 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:50.215605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8753e-310d-40fe-a0cb-09d4f32d8359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:50.216239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8753e-310d-40fe-a0cb-09d4f32d8359 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:50.216519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8753e-310d-40fe-a0cb-09d4f32d8359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:50.216791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8753e-310d-40fe-a0cb-09d4f32d8359 admin admin] PATH_INFO: `/v3/projects/f3929898cf764cd6a05e8662e83d95e0` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:50.223801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8753e-310d-40fe-a0cb-09d4f32d8359 admin admin] RBAC: Authorizing `identity:get_project(project_id=f3929898cf764cd6a05e8662e83d95e0)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:50.225601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8753e-310d-40fe-a0cb-09d4f32d8359 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:50.227290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 146/295] 10.222.0.42 () {56 vars in 1207 bytes} [Mon Oct 24 12:59:50 2022] GET /identity/v3/projects/f3929898cf764cd6a05e8662e83d95e0 => generated 344 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:50.234828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bc734-1b64-4948-8882-e5f163d78bc9 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:50.251464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8bc734-1b64-4948-8882-e5f163d78bc9 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:50.255979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb7138-43b9-44cd-b947-478a1a3eea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:50.256998 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb7138-43b9-44cd-b947-478a1a3eea94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:50.257283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb7138-43b9-44cd-b947-478a1a3eea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:50.257559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb7138-43b9-44cd-b947-478a1a3eea94 admin admin] PATH_INFO: `/v3/projects/f3929898cf764cd6a05e8662e83d95e0/groups/1cf2dbe75dfa4b3ca970768c8f1a52a8/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:50.261717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb7138-43b9-44cd-b947-478a1a3eea94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3929898cf764cd6a05e8662e83d95e0, group_id=1cf2dbe75dfa4b3ca970768c8f1a52a8, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:50.264705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb7138-43b9-44cd-b947-478a1a3eea94 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:50.287477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 150/296] 10.222.0.42 () {58 vars in 1463 bytes} [Mon Oct 24 12:59:50 2022] PUT /identity/v3/projects/f3929898cf764cd6a05e8662e83d95e0/groups/1cf2dbe75dfa4b3ca970768c8f1a52a8/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 12:59:51.534540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8f47d-ac0f-494f-9297-152ec3a91504 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:51.535402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8f47d-ac0f-494f-9297-152ec3a91504 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:51.535908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba8f47d-ac0f-494f-9297-152ec3a91504 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:51.537800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 147/297] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:51.543868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f7373-2030-4290-bf45-3a7abc6fc7f2 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:51.544283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f7373-2030-4290-bf45-3a7abc6fc7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:51.544411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f7373-2030-4290-bf45-3a7abc6fc7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:51.572749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-049f7373-2030-4290-bf45-3a7abc6fc7f2 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:51.574986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-049f7373-2030-4290-bf45-3a7abc6fc7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:51.586786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd80b9-c792-4fab-9f92-62f04fc68a16 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:51.587447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd80b9-c792-4fab-9f92-62f04fc68a16 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:51.587930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd80b9-c792-4fab-9f92-62f04fc68a16 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:51.590304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 148/298] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:51.595776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b22e9-a601-4ab8-b0a0-322116eae7b4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:51.596311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b22e9-a601-4ab8-b0a0-322116eae7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:51.596787 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399b22e9-a601-4ab8-b0a0-322116eae7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:51.631546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-399b22e9-a601-4ab8-b0a0-322116eae7b4 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:51.632744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-399b22e9-a601-4ab8-b0a0-322116eae7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:51.666178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:51.690369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 151/299] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:51.735889 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:51.740789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 149/300] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:51.986302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23d28-6370-463f-b18e-2e6faa3821b8 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:51.986824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23d28-6370-463f-b18e-2e6faa3821b8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:51.987268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e23d28-6370-463f-b18e-2e6faa3821b8 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:51.989033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 152/301] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:51.994549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7793a-f320-4126-b428-3403aa0a298b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:51.995117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7793a-f320-4126-b428-3403aa0a298b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:51.995595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7793a-f320-4126-b428-3403aa0a298b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:52.017089 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2fc7793a-f320-4126-b428-3403aa0a298b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:52.018618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2fc7793a-f320-4126-b428-3403aa0a298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:52.098313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 150/302] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:52.112695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e23d28-6370-463f-b18e-2e6faa3821b8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:52.144557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7793a-f320-4126-b428-3403aa0a298b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:52.235832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e23d28-6370-463f-b18e-2e6faa3821b8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:52.267284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7793a-f320-4126-b428-3403aa0a298b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:52.276135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02982db6-7abd-4c28-afe3-07e81544f5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:52.277033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02982db6-7abd-4c28-afe3-07e81544f5ef None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:52.277464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02982db6-7abd-4c28-afe3-07e81544f5ef None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:52.277988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02982db6-7abd-4c28-afe3-07e81544f5ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:52.283629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:52.293616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02982db6-7abd-4c28-afe3-07e81544f5ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-02982db6-7abd-4c28-afe3-07e81544f5ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:52.299455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:52.300525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:52.301446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:52.303877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 153/303] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 12:59:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:52.309744 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02982db6-7abd-4c28-afe3-07e81544f5ef None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:52.311331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad4ed9-acf1-497f-bb1b-8cc5fad82e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:52.314366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad4ed9-acf1-497f-bb1b-8cc5fad82e99 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:52.314812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad4ed9-acf1-497f-bb1b-8cc5fad82e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:52.315387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad4ed9-acf1-497f-bb1b-8cc5fad82e99 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:52.328508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad4ed9-acf1-497f-bb1b-8cc5fad82e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-17ad4ed9-acf1-497f-bb1b-8cc5fad82e99 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:52.334060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:52.335122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:52.335945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:52.338194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 151/304] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 12:59:52.345244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad4ed9-acf1-497f-bb1b-8cc5fad82e99 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:52.372718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:52.442814 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02982db6-7abd-4c28-afe3-07e81544f5ef None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:52.470499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad4ed9-acf1-497f-bb1b-8cc5fad82e99 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:52.492612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:52.493234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:52.493517 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:52.493859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:52.494456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:52.494749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:52.496872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:52.519180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 154/305] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 12:59:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:52.525995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:52.542206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:52.542894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:52.543213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:52.543496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:52.544175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:52.544483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:52.545866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:52.566753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 152/306] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:52.572622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:52.620903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:52.680333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134cdd2-8349-45a3-90f4-9ec92b6a4a2e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:52.695943 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e5e5-0858-4dd7-9ffa-1d3896857483 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:52.697109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:52.722779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece4dfa-9706-4969-bbaa-7ac58c5f94a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:52.723778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4dfa-9706-4969-bbaa-7ac58c5f94a9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:52.724198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4dfa-9706-4969-bbaa-7ac58c5f94a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:52.724612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece4dfa-9706-4969-bbaa-7ac58c5f94a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:52.741598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857c6e-42fc-4e26-b17c-097283abba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:52.742225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857c6e-42fc-4e26-b17c-097283abba7c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:52.742511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857c6e-42fc-4e26-b17c-097283abba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:52.742788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857c6e-42fc-4e26-b17c-097283abba7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:52.807988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece4dfa-9706-4969-bbaa-7ac58c5f94a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:52.809820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece4dfa-9706-4969-bbaa-7ac58c5f94a9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:52.832950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2857c6e-42fc-4e26-b17c-097283abba7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:52.834535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2857c6e-42fc-4e26-b17c-097283abba7c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:52.877757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:52.926899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 155/307] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:52.928266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece4dfa-9706-4969-bbaa-7ac58c5f94a9 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:52.948853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:52.963636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 153/308] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 12:59:52.965378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857c6e-42fc-4e26-b17c-097283abba7c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.052061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece4dfa-9706-4969-bbaa-7ac58c5f94a9 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:53.080892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857c6e-42fc-4e26-b17c-097283abba7c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:53.094831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba2482-7990-4f96-a6b2-2f8670751121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:53.095507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba2482-7990-4f96-a6b2-2f8670751121 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.095791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba2482-7990-4f96-a6b2-2f8670751121 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.096077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba2482-7990-4f96-a6b2-2f8670751121 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.107012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba2482-7990-4f96-a6b2-2f8670751121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-72ba2482-7990-4f96-a6b2-2f8670751121 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:53.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:53.113058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 24 12:59:53.113851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:53.115875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 156/309] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:53.116804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba2482-7990-4f96-a6b2-2f8670751121 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.129611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a4fb7-fe54-43c7-876c-b78e32fc514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:53.130261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a4fb7-fe54-43c7-876c-b78e32fc514d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.130856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a4fb7-fe54-43c7-876c-b78e32fc514d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.131425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a4fb7-fe54-43c7-876c-b78e32fc514d None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.132619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a4fb7-fe54-43c7-876c-b78e32fc514d None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:53.134615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a4fb7-fe54-43c7-876c-b78e32fc514d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:53.137430 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 154/310] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:52 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:53.138808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fdfcf-2045-4168-a7f3-be5c5fc25481 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.139295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fdfcf-2045-4168-a7f3-be5c5fc25481 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.139749 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fdfcf-2045-4168-a7f3-be5c5fc25481 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.141178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 155/311] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:53.142069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fdfcf-2045-4168-a7f3-be5c5fc25481 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.147570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:53.210046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:53.248370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba2482-7990-4f96-a6b2-2f8670751121 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:53.294699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fdfcf-2045-4168-a7f3-be5c5fc25481 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:53.303286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18c18a-5765-427b-8038-837a2f1cf554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:53.304039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18c18a-5765-427b-8038-837a2f1cf554 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.304392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18c18a-5765-427b-8038-837a2f1cf554 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.304730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad18c18a-5765-427b-8038-837a2f1cf554 None admin] PATH_INFO: `/v3/projects/1634123c90c5493296570578db87c5c2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.307504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18c18a-5765-427b-8038-837a2f1cf554 None admin] RBAC: Authorizing `identity:get_project(project_id=1634123c90c5493296570578db87c5c2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:53.308545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad18c18a-5765-427b-8038-837a2f1cf554 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:53.312690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 157/312] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:53 2022] GET /identity/v3/projects/1634123c90c5493296570578db87c5c2 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:53.312690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18c18a-5765-427b-8038-837a2f1cf554 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.335029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193ed0d-6d4c-4041-8d5d-921b50b02c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:53.335612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ed0d-6d4c-4041-8d5d-921b50b02c09 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.335899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ed0d-6d4c-4041-8d5d-921b50b02c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.336179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193ed0d-6d4c-4041-8d5d-921b50b02c09 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.336853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193ed0d-6d4c-4041-8d5d-921b50b02c09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:53.337160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193ed0d-6d4c-4041-8d5d-921b50b02c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:53.338534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193ed0d-6d4c-4041-8d5d-921b50b02c09 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:53.350639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 156/313] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 12:59:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:53.351750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57380dbb-424e-47b5-9119-665694065fcb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.352334 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57380dbb-424e-47b5-9119-665694065fcb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.352853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57380dbb-424e-47b5-9119-665694065fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.373919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-57380dbb-424e-47b5-9119-665694065fcb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:53.375472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-57380dbb-424e-47b5-9119-665694065fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:53.433973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad18c18a-5765-427b-8038-837a2f1cf554 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:53.456673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:53.469108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 157/314] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:53.471176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57380dbb-424e-47b5-9119-665694065fcb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.488030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12094551-7be6-4ca5-813e-166fe840c418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:53.489126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12094551-7be6-4ca5-813e-166fe840c418 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.489953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12094551-7be6-4ca5-813e-166fe840c418 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.490415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12094551-7be6-4ca5-813e-166fe840c418 None admin] PATH_INFO: `/v3/groups/46b7dac9dfb84e58baf059cba923fbeb` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.491995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12094551-7be6-4ca5-813e-166fe840c418 None admin] RBAC: Authorizing `identity:get_group(group_id=46b7dac9dfb84e58baf059cba923fbeb)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:53.494065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12094551-7be6-4ca5-813e-166fe840c418 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:53.497249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 158/315] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 24 12:59:53 2022] GET /identity/v3/groups/46b7dac9dfb84e58baf059cba923fbeb => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:53.498250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12094551-7be6-4ca5-813e-166fe840c418 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.520513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:53.605228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57380dbb-424e-47b5-9119-665694065fcb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:53.627887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12094551-7be6-4ca5-813e-166fe840c418 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:53.643612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecc43f-c54f-4efe-93a1-ab7df167dc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:53.644250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecc43f-c54f-4efe-93a1-ab7df167dc05 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.644529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecc43f-c54f-4efe-93a1-ab7df167dc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.644803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ecc43f-c54f-4efe-93a1-ab7df167dc05 None admin] PATH_INFO: `/v3/groups/46b7dac9dfb84e58baf059cba923fbeb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.645892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecc43f-c54f-4efe-93a1-ab7df167dc05 None admin] RBAC: Authorizing `identity:get_group(group_id=46b7dac9dfb84e58baf059cba923fbeb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:53.647362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ecc43f-c54f-4efe-93a1-ab7df167dc05 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:53.649099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 158/316] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 24 12:59:53 2022] GET /identity/v3/groups/46b7dac9dfb84e58baf059cba923fbeb => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:53.649988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecc43f-c54f-4efe-93a1-ab7df167dc05 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.670269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f050-bc5d-4522-a45b-98c73be3f569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:53.671045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f050-bc5d-4522-a45b-98c73be3f569 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.671373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f050-bc5d-4522-a45b-98c73be3f569 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.671723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f050-bc5d-4522-a45b-98c73be3f569 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.735501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:53.766724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f050-bc5d-4522-a45b-98c73be3f569 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:53.768447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f050-bc5d-4522-a45b-98c73be3f569 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:53.784528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:53.785727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ecc43f-c54f-4efe-93a1-ab7df167dc05 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:53.831983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9d75a5-b8cb-4775-a42c-11f69af6329b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:53.832599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d75a5-b8cb-4775-a42c-11f69af6329b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.832882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d75a5-b8cb-4775-a42c-11f69af6329b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.833157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9d75a5-b8cb-4775-a42c-11f69af6329b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.833901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d75a5-b8cb-4775-a42c-11f69af6329b None admin] RBAC: Adding query filter params (group.id=46b7dac9dfb84e58baf059cba923fbeb, role.id=78ff1dcdc39743048089711fcfc4d3a4, scope.project.id=1634123c90c5493296570578db87c5c2) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:53.834214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d75a5-b8cb-4775-a42c-11f69af6329b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:53.836542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9d75a5-b8cb-4775-a42c-11f69af6329b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:53.843298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 159/317] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 24 12:59:53 2022] GET /identity/v3/role_assignments?group.id=46b7dac9dfb84e58baf059cba923fbeb&scope.project.id=1634123c90c5493296570578db87c5c2&role.id=78ff1dcdc39743048089711fcfc4d3a4 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:53.844494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719f539-6873-4b56-bc6d-191595dba30a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:53.844771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719f539-6873-4b56-bc6d-191595dba30a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:53.845041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719f539-6873-4b56-bc6d-191595dba30a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:53.850391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 160/318] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:53.850391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719f539-6873-4b56-bc6d-191595dba30a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.903811 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 159/319] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:53.904783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f050-bc5d-4522-a45b-98c73be3f569 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:53.993323 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719f539-6873-4b56-bc6d-191595dba30a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.027940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:54.040163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f7d47-4457-4f2e-860d-ad348841d2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.040936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f7d47-4457-4f2e-860d-ad348841d2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.041224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f7d47-4457-4f2e-860d-ad348841d2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.041503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f7d47-4457-4f2e-860d-ad348841d2f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.042306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f7d47-4457-4f2e-860d-ad348841d2f7 None admin] RBAC: Adding query filter params (group.id=46b7dac9dfb84e58baf059cba923fbeb, role.id=54e93caca0fb4c59a01ae64a0ed3a610, scope.project.id=1634123c90c5493296570578db87c5c2) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:54.042615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f7d47-4457-4f2e-860d-ad348841d2f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:54.044907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f7d47-4457-4f2e-860d-ad348841d2f7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:54.047242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f050-bc5d-4522-a45b-98c73be3f569 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.051948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 161/320] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 24 12:59:53 2022] GET /identity/v3/role_assignments?group.id=46b7dac9dfb84e58baf059cba923fbeb&scope.project.id=1634123c90c5493296570578db87c5c2&role.id=54e93caca0fb4c59a01ae64a0ed3a610 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:54.053129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f31d-24b0-4129-b7f5-8f6e2f681eed None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.053452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f31d-24b0-4129-b7f5-8f6e2f681eed None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.053738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3f31d-24b0-4129-b7f5-8f6e2f681eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.060857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:54.078911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6cb3f31d-24b0-4129-b7f5-8f6e2f681eed None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:54.080206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6cb3f31d-24b0-4129-b7f5-8f6e2f681eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:54.101575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a8d00-2d01-43b8-b72f-0153b08d945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.102236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a8d00-2d01-43b8-b72f-0153b08d945a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.102518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a8d00-2d01-43b8-b72f-0153b08d945a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.102792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a8d00-2d01-43b8-b72f-0153b08d945a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.116032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877a8d00-2d01-43b8-b72f-0153b08d945a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-877a8d00-2d01-43b8-b72f-0153b08d945a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:54.122724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:54.123833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 12:59:54.124611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:54.127446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 160/321] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 12:59:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:54.133155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a8d00-2d01-43b8-b72f-0153b08d945a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.177716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 162/322] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:54.178376 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f31d-24b0-4129-b7f5-8f6e2f681eed None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.287633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a8d00-2d01-43b8-b72f-0153b08d945a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.302279 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3f31d-24b0-4129-b7f5-8f6e2f681eed None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.330183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:54.333547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b9c98-9110-47fc-9a86-f940babac58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.334188 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b9c98-9110-47fc-9a86-f940babac58f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.334466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b9c98-9110-47fc-9a86-f940babac58f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.334738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b9c98-9110-47fc-9a86-f940babac58f None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.336835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b9c98-9110-47fc-9a86-f940babac58f None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:54.338289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b9c98-9110-47fc-9a86-f940babac58f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:54.341083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 161/323] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:54.344846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.345660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.346073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.346488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.347008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b9c98-9110-47fc-9a86-f940babac58f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.347518 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:54.347980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:54.349494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:54.364134 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 163/324] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:54.369910 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.386394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:54.473284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b9c98-9110-47fc-9a86-f940babac58f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.501247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf188e4-135d-42d6-9101-caae05b61566 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.512482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65559bc0-8398-4c7d-a42e-ea086af3827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.513130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65559bc0-8398-4c7d-a42e-ea086af3827d None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.513424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65559bc0-8398-4c7d-a42e-ea086af3827d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.513706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65559bc0-8398-4c7d-a42e-ea086af3827d None admin] PATH_INFO: `/v3/groups/46b7dac9dfb84e58baf059cba923fbeb` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.515150 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65559bc0-8398-4c7d-a42e-ea086af3827d None admin] RBAC: Authorizing `identity:get_group(group_id=46b7dac9dfb84e58baf059cba923fbeb)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:54.516610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65559bc0-8398-4c7d-a42e-ea086af3827d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:54.518244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 162/325] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/groups/46b7dac9dfb84e58baf059cba923fbeb => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:54.519112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65559bc0-8398-4c7d-a42e-ea086af3827d None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.540312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef1ca0-6d70-4bc2-af80-a23e5dbc9522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.540927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef1ca0-6d70-4bc2-af80-a23e5dbc9522 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.541210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef1ca0-6d70-4bc2-af80-a23e5dbc9522 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.541487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ef1ca0-6d70-4bc2-af80-a23e5dbc9522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.606285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:54.629275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef1ca0-6d70-4bc2-af80-a23e5dbc9522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:54.631532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ef1ca0-6d70-4bc2-af80-a23e5dbc9522 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:54.634543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65559bc0-8398-4c7d-a42e-ea086af3827d None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.638511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:54.676034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b05799-c91c-497d-ab02-29a19a4673f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.676673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b05799-c91c-497d-ab02-29a19a4673f8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.676953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b05799-c91c-497d-ab02-29a19a4673f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.677225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b05799-c91c-497d-ab02-29a19a4673f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.688588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b05799-c91c-497d-ab02-29a19a4673f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-97b05799-c91c-497d-ab02-29a19a4673f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 12:59:54.693506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:54.694377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:54.695213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 163/326] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:54.695863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b05799-c91c-497d-ab02-29a19a4673f8 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.759731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 164/327] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 24 12:59:54.761219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef1ca0-6d70-4bc2-af80-a23e5dbc9522 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.816986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b05799-c91c-497d-ab02-29a19a4673f8 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.848801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:54.850458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.851080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.851371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.851647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.852372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] RBAC: Adding query filter params (group.id=46b7dac9dfb84e58baf059cba923fbeb, role.id=54e93caca0fb4c59a01ae64a0ed3a610, scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:54.852688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:54.854112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:54.860076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 164/328] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/role_assignments?group.id=46b7dac9dfb84e58baf059cba923fbeb&scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd&role.id=54e93caca0fb4c59a01ae64a0ed3a610 => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:54.860974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.864769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ef1ca0-6d70-4bc2-af80-a23e5dbc9522 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:54.898974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:54.905197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:54.905814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:54.906197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:54.906397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:54.907729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:54.907729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:54.908733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:54.918887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 165/329] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:54.924649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:54.983511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261d7eb5-c91e-481c-b0a4-a0cb5000526f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:55.024606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465b28f-5be5-4e3b-acff-2623fc0d7116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:55.025170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465b28f-5be5-4e3b-acff-2623fc0d7116 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:55.025452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465b28f-5be5-4e3b-acff-2623fc0d7116 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:55.025836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465b28f-5be5-4e3b-acff-2623fc0d7116 None admin] PATH_INFO: `/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:55.027630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465b28f-5be5-4e3b-acff-2623fc0d7116 None admin] RBAC: Authorizing `identity:get_project(project_id=97d9afe79c47436899dad4fbd1cd7bfd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:55.029070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465b28f-5be5-4e3b-acff-2623fc0d7116 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:55.030729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 165/330] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/projects/97d9afe79c47436899dad4fbd1cd7bfd => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:55.037606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465b28f-5be5-4e3b-acff-2623fc0d7116 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:55.049888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc512f-66d4-4d30-82d0-cb119dcb2408 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:55.097636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cdb65-f2ac-4a63-8a72-5198dff09777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:55.098362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cdb65-f2ac-4a63-8a72-5198dff09777 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:55.098747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cdb65-f2ac-4a63-8a72-5198dff09777 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:55.099113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cdb65-f2ac-4a63-8a72-5198dff09777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:55.117367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:55.152363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:55.159118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465b28f-5be5-4e3b-acff-2623fc0d7116 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:55.185335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cdb65-f2ac-4a63-8a72-5198dff09777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:55.187704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271cdb65-f2ac-4a63-8a72-5198dff09777 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:55.207755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7d8da-e8ca-4001-b0c2-d095e4156e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:55.208504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7d8da-e8ca-4001-b0c2-d095e4156e3e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:55.208785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7d8da-e8ca-4001-b0c2-d095e4156e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:55.209053 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7d8da-e8ca-4001-b0c2-d095e4156e3e None admin] PATH_INFO: `/v3/groups/46b7dac9dfb84e58baf059cba923fbeb` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:55.212115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7d8da-e8ca-4001-b0c2-d095e4156e3e None admin] RBAC: Authorizing `identity:get_group(group_id=46b7dac9dfb84e58baf059cba923fbeb)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:55.212678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7d8da-e8ca-4001-b0c2-d095e4156e3e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:55.214406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 166/331] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 24 12:59:55 2022] GET /identity/v3/groups/46b7dac9dfb84e58baf059cba923fbeb => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:55.220116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7d8da-e8ca-4001-b0c2-d095e4156e3e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:55.301542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 166/332] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 12:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 12:59:55.307148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cdb65-f2ac-4a63-8a72-5198dff09777 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:55.327695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7d8da-e8ca-4001-b0c2-d095e4156e3e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:55.351517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:55.361882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:55.362488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:55.362771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:55.363048 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:55.363795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] RBAC: Adding query filter params (group.id=46b7dac9dfb84e58baf059cba923fbeb, role.id=78ff1dcdc39743048089711fcfc4d3a4, scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:55.364105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:55.365528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:55.371701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 167/333] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 24 12:59:55 2022] GET /identity/v3/role_assignments?group.id=46b7dac9dfb84e58baf059cba923fbeb&scope.project.id=97d9afe79c47436899dad4fbd1cd7bfd&role.id=78ff1dcdc39743048089711fcfc4d3a4 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:55.419109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271cdb65-f2ac-4a63-8a72-5198dff09777 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:55.456559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec0b1-901b-436d-b91c-b6e7de70dcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:55.457197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec0b1-901b-436d-b91c-b6e7de70dcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:55.457476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec0b1-901b-436d-b91c-b6e7de70dcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:55.457748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec0b1-901b-436d-b91c-b6e7de70dcd8 None admin] PATH_INFO: `/v3/projects/f3929898cf764cd6a05e8662e83d95e0` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:55.458898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eec0b1-901b-436d-b91c-b6e7de70dcd8 None admin] RBAC: Authorizing `identity:get_project(project_id=f3929898cf764cd6a05e8662e83d95e0)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:55.460384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eec0b1-901b-436d-b91c-b6e7de70dcd8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:55.462089 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 167/334] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 12:59:55 2022] GET /identity/v3/projects/f3929898cf764cd6a05e8662e83d95e0 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:55.468532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:55.500216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:55.574730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47da5a-1790-4815-8505-23a498367339 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:55.608947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f201e0-1211-473e-884b-ca98f201bf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:55.609578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f201e0-1211-473e-884b-ca98f201bf8e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:55.609861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f201e0-1211-473e-884b-ca98f201bf8e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:55.610134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f201e0-1211-473e-884b-ca98f201bf8e None admin] PATH_INFO: `/v3/groups/1cf2dbe75dfa4b3ca970768c8f1a52a8` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:55.611260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f201e0-1211-473e-884b-ca98f201bf8e None admin] RBAC: Authorizing `identity:get_group(group_id=1cf2dbe75dfa4b3ca970768c8f1a52a8)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:55.612687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f201e0-1211-473e-884b-ca98f201bf8e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:55.614271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 168/335] 10.222.0.42 () {56 vars in 1180 bytes} [Mon Oct 24 12:59:55 2022] GET /identity/v3/groups/1cf2dbe75dfa4b3ca970768c8f1a52a8 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:55.620218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec0b1-901b-436d-b91c-b6e7de70dcd8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:55.723240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec0b1-901b-436d-b91c-b6e7de70dcd8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:55.737114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:55.758158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837492e-8e5f-4d4f-ada5-732f9d84d776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:55.758799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837492e-8e5f-4d4f-ada5-732f9d84d776 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:55.759110 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837492e-8e5f-4d4f-ada5-732f9d84d776 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:55.759389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837492e-8e5f-4d4f-ada5-732f9d84d776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:55.760116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837492e-8e5f-4d4f-ada5-732f9d84d776 None admin] RBAC: Adding query filter params (group.id=1cf2dbe75dfa4b3ca970768c8f1a52a8, role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=f3929898cf764cd6a05e8662e83d95e0) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:55.760426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837492e-8e5f-4d4f-ada5-732f9d84d776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:55.762583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837492e-8e5f-4d4f-ada5-732f9d84d776 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:55.768183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 168/336] 10.222.0.42 () {56 vars in 1376 bytes} [Mon Oct 24 12:59:55 2022] GET /identity/v3/role_assignments?group.id=1cf2dbe75dfa4b3ca970768c8f1a52a8&scope.project.id=f3929898cf764cd6a05e8662e83d95e0&role.id=11b74890c7484c6088dd78c106de8be4 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:58.435231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668119e-4c91-429f-8db9-f9dab6b625fb None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:58.436131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668119e-4c91-429f-8db9-f9dab6b625fb None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:58.436563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668119e-4c91-429f-8db9-f9dab6b625fb None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:58.438398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 169/337] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:58.443358 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c3c9-6b14-4487-b430-6735f3af6882 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:58.444108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c3c9-6b14-4487-b430-6735f3af6882 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:58.444538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292c3c9-6b14-4487-b430-6735f3af6882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:58.468315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4292c3c9-6b14-4487-b430-6735f3af6882 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:58.469880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4292c3c9-6b14-4487-b430-6735f3af6882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:58.567434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 169/338] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:58.661693 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c19f10-5ccc-4921-b677-3105b52892fb None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:58.662983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c19f10-5ccc-4921-b677-3105b52892fb None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:58.663731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c19f10-5ccc-4921-b677-3105b52892fb None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:58.665529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 170/339] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:58.679274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a016bd-d676-4770-b47c-4b90ea79a906 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:58.681423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a016bd-d676-4770-b47c-4b90ea79a906 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:58.681423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a016bd-d676-4770-b47c-4b90ea79a906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:58.707669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-77a016bd-d676-4770-b47c-4b90ea79a906 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:58.708772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-77a016bd-d676-4770-b47c-4b90ea79a906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:58.809054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 170/340] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:58.926812 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c19f10-5ccc-4921-b677-3105b52892fb None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:59.082996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:59.096695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c19f10-5ccc-4921-b677-3105b52892fb None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:59.112124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a436a-a55c-409f-abe4-8f3fe4849eb5 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.112822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a436a-a55c-409f-abe4-8f3fe4849eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.113331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a436a-a55c-409f-abe4-8f3fe4849eb5 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.119357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 171/341] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 12:59:59.122264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bee68-9458-4b7f-800c-855d3051d5d7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.123248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bee68-9458-4b7f-800c-855d3051d5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.123760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666bee68-9458-4b7f-800c-855d3051d5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.148515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-666bee68-9458-4b7f-800c-855d3051d5d7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:59.151777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-666bee68-9458-4b7f-800c-855d3051d5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:59.171362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:59.178066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.179676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.180245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.203920 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:59.205748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:59.206952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 12:59:59.226774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:59.228231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 12:59:59.229111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 12:59:59.229111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:59.229111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:59.234072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 171/342] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:59.239713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:59.291081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 172/343] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 12:59:59.291081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bee68-9458-4b7f-800c-855d3051d5d7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:59.444524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65144e4-531b-4a75-8318-54089b453334 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:59.511453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3fef45-9b78-4bf0-85f7-c0786c8d53c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:59.512265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fef45-9b78-4bf0-85f7-c0786c8d53c0 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.512549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fef45-9b78-4bf0-85f7-c0786c8d53c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.512829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fef45-9b78-4bf0-85f7-c0786c8d53c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.513847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fef45-9b78-4bf0-85f7-c0786c8d53c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:59.517288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fef45-9b78-4bf0-85f7-c0786c8d53c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:59.518755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3fef45-9b78-4bf0-85f7-c0786c8d53c0 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:59.524543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666bee68-9458-4b7f-800c-855d3051d5d7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:59.538746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 172/344] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 12:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 12:59:59.540130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f144e9-f1f7-49ac-9119-1bc59ab13f6f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.540319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f144e9-f1f7-49ac-9119-1bc59ab13f6f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.540602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f144e9-f1f7-49ac-9119-1bc59ab13f6f None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.547367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 173/345] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 12:59:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 12:59:59.548194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f144e9-f1f7-49ac-9119-1bc59ab13f6f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:59.562057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:59.575246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02eda2d-b898-43d5-80b2-43dd264ef871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:59.576258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eda2d-b898-43d5-80b2-43dd264ef871 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.576833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eda2d-b898-43d5-80b2-43dd264ef871 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.577299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eda2d-b898-43d5-80b2-43dd264ef871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.591185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02eda2d-b898-43d5-80b2-43dd264ef871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:59.593033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02eda2d-b898-43d5-80b2-43dd264ef871 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f02eda2d-b898-43d5-80b2-43dd264ef871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 12:59:59.614054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:59.615032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 12:59:59.615924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 12:59:59.615924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:59.615924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 12:59:59.619136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 173/346] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:59.620796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913d343-cc70-49c7-ae22-cc1765276441 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.621346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913d343-cc70-49c7-ae22-cc1765276441 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.621823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913d343-cc70-49c7-ae22-cc1765276441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.642057 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6913d343-cc70-49c7-ae22-cc1765276441 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 12:59:59.643287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6913d343-cc70-49c7-ae22-cc1765276441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 12:59:59.658727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:59.665406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f144e9-f1f7-49ac-9119-1bc59ab13f6f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:59.702845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cf8c3-45ec-444a-ba25-27bc1eefbcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:59.703693 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cf8c3-45ec-444a-ba25-27bc1eefbcf9 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.703980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cf8c3-45ec-444a-ba25-27bc1eefbcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.704420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cf8c3-45ec-444a-ba25-27bc1eefbcf9 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.705405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cf8c3-45ec-444a-ba25-27bc1eefbcf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:59.707089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768cf8c3-45ec-444a-ba25-27bc1eefbcf9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:59.723130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 174/347] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 12:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 12:59:59.723130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913d343-cc70-49c7-ae22-cc1765276441 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:59.740029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 174/348] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:59 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 12:59:59.741420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cf8c3-45ec-444a-ba25-27bc1eefbcf9 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:59.823987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 12:59:59.864708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913d343-cc70-49c7-ae22-cc1765276441 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:59.868010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cf8c3-45ec-444a-ba25-27bc1eefbcf9 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 12:59:59.914068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:59.914494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 12:59:59.914721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.915005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.915323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.915884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 12:59:59.916339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 12:59:59.916877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 12:59:59.917868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 12:59:59.918363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:59.919952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:59.929447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 12:59:59.931044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 12:59:59.946263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 175/349] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 12:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 12:59:59.951131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 12:59:59.952204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 12:59:59.953368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 12:59:59.953368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 12:59:59.953368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 12:59:59.953368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 175/350] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 12:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 12:59:59.954691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:59.959169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 12:59:59.982011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:00.093180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3aecb-7520-430a-a7f3-4e87d728904a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:00.098398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ad15-9c66-495e-9714-d08d799b068c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:00.115847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:00.132596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5185b1f-d0a0-455f-94da-105688274413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:00.133208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5185b1f-d0a0-455f-94da-105688274413 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:00.133492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5185b1f-d0a0-455f-94da-105688274413 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:00.133768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5185b1f-d0a0-455f-94da-105688274413 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:00.134429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5185b1f-d0a0-455f-94da-105688274413 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:00.136441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed8535-0797-42af-9a76-347af5a09737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:00.136763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5185b1f-d0a0-455f-94da-105688274413 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:00.137050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8535-0797-42af-9a76-347af5a09737 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:00.137332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8535-0797-42af-9a76-347af5a09737 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:00.137606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed8535-0797-42af-9a76-347af5a09737 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:00.138249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed8535-0797-42af-9a76-347af5a09737 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:00.138556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed8535-0797-42af-9a76-347af5a09737 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:00.139951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed8535-0797-42af-9a76-347af5a09737 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:00.153345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 176/351] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 12:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:00.154444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d172-ed85-4870-90fb-a954ca87541a None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:00.154732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d172-ed85-4870-90fb-a954ca87541a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:00.155007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d172-ed85-4870-90fb-a954ca87541a None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:00.156418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 177/352] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:00.160783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d172-ed85-4870-90fb-a954ca87541a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:00.163820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 176/353] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 12:59:59 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:00.164507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5185b1f-d0a0-455f-94da-105688274413 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:00.242130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:00.292802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5185b1f-d0a0-455f-94da-105688274413 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:00.331336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8816da2-6b54-441b-923e-2c0e909edf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:00.332031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8816da2-6b54-441b-923e-2c0e909edf32 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:00.332322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8816da2-6b54-441b-923e-2c0e909edf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:00.332605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8816da2-6b54-441b-923e-2c0e909edf32 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:00.333325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8816da2-6b54-441b-923e-2c0e909edf32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:00.334813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8816da2-6b54-441b-923e-2c0e909edf32 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:00.336115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d172-ed85-4870-90fb-a954ca87541a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:00.363342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 177/354] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:00:00 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:00.364603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f473cb3-dbc8-4443-9ca5-de188543f97e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:00.364949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f473cb3-dbc8-4443-9ca5-de188543f97e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:00.365253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f473cb3-dbc8-4443-9ca5-de188543f97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:00.391514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6f473cb3-dbc8-4443-9ca5-de188543f97e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:00.392788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6f473cb3-dbc8-4443-9ca5-de188543f97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:00.392973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:00.393938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:00.394323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:00.394667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:00.407939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:00.409640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:00.429027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:00.430008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:00.431412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:00.431412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:00.431412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:00.431412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 178/355] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:00.437749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:00.477452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 178/356] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:00.477965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:00.563295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca430e90-333c-46b4-a549-085d570d95e1 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:00.604637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:00.605584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:00.606226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:00.606689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:00.607546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:00.608040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:00.609605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:00.623003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 179/357] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:00.628326 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f473cb3-dbc8-4443-9ca5-de188543f97e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:00.701825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:00.766833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f473cb3-dbc8-4443-9ca5-de188543f97e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:00.772281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:00.802509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34afd5-f3fb-40e1-b760-11cabd69b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:00.803152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34afd5-f3fb-40e1-b760-11cabd69b16f None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:00.803442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34afd5-f3fb-40e1-b760-11cabd69b16f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:00.803722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34afd5-f3fb-40e1-b760-11cabd69b16f None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:00.804394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34afd5-f3fb-40e1-b760-11cabd69b16f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:00.805853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34afd5-f3fb-40e1-b760-11cabd69b16f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:00.832471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 179/358] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:00:00 2022] POST /identity/v3/users => generated 249 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:00.954212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:00.970981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb98d0-0e04-4565-8b2b-a68fc3310559 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:01.021569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:01.022184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:01.022466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:01.022741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:01.023513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:01.024870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:01.034287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.sql.core [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:00:01.034287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 24 13:00:01.034287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [parameters: {'id': 'a8f641be45df4fe29817df5ff2d4efd3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 24 13:00:01.034287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81614) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result.read() Oct 24 13:00:01.051086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return ref.to_dict() Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:00:01.052054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application yield resource Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise self.value Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application session.commit() Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:00:01.053637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:00:01.054519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:00:01.055414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result.read() Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:00:01.056214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [parameters: {'id': 'a8f641be45df4fe29817df5ff2d4efd3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:01.059437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 24 13:00:01.061594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:01.064010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 180/359] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 24 13:00:00 2022] POST /identity/v3/roles => generated 144 bytes in 281 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:01.064145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34afd5-f3fb-40e1-b760-11cabd69b16f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:01.177842 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34afd5-f3fb-40e1-b760-11cabd69b16f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:01.190575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:01.229708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bea923-4682-4dae-a03c-81ae9458304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:01.230358 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea923-4682-4dae-a03c-81ae9458304d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:01.230641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea923-4682-4dae-a03c-81ae9458304d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:01.230917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bea923-4682-4dae-a03c-81ae9458304d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:01.245965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bea923-4682-4dae-a03c-81ae9458304d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-07bea923-4682-4dae-a03c-81ae9458304d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:01.252126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:01.253029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 24 13:00:01.253817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:01.255868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 180/360] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:01.261354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:01.397973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bcd33-2f7b-474e-bec1-c02a0c61393d None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:01.452833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7b6dc-2909-4f96-9e7f-994d327138f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:01.453388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7b6dc-2909-4f96-9e7f-994d327138f5 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:01.453675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7b6dc-2909-4f96-9e7f-994d327138f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:01.453943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7b6dc-2909-4f96-9e7f-994d327138f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:01.454691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7b6dc-2909-4f96-9e7f-994d327138f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:01.455000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7b6dc-2909-4f96-9e7f-994d327138f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:01.456407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7b6dc-2909-4f96-9e7f-994d327138f5 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:01.467352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 181/361] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:02.420088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6dd49-7062-4370-857a-27cf99cb09b6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:02.420773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6dd49-7062-4370-857a-27cf99cb09b6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:02.420773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6dd49-7062-4370-857a-27cf99cb09b6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:02.422246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 181/362] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:02.427509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13537e5-1cc9-4393-af7c-0b1c5f7b3552 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:02.427966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13537e5-1cc9-4393-af7c-0b1c5f7b3552 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:02.428125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13537e5-1cc9-4393-af7c-0b1c5f7b3552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:02.455440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d13537e5-1cc9-4393-af7c-0b1c5f7b3552 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:02.456634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d13537e5-1cc9-4393-af7c-0b1c5f7b3552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:02.467706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:02.553404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 182/363] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:02.942980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6dd49-7062-4370-857a-27cf99cb09b6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:03.079090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6dd49-7062-4370-857a-27cf99cb09b6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:03.128810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a7abe-d3c4-48a9-a681-e52c463b2880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:03.129558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad687f7-72af-4496-9b05-e54caf40ec0c None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.129844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad687f7-72af-4496-9b05-e54caf40ec0c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.130116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad687f7-72af-4496-9b05-e54caf40ec0c None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.130475 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a7abe-d3c4-48a9-a681-e52c463b2880 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.130949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a7abe-d3c4-48a9-a681-e52c463b2880 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.131760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a7abe-d3c4-48a9-a681-e52c463b2880 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.131983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 183/364] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:03.137323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba601a45-2a7d-4641-b714-19ae0640e13b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.137613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba601a45-2a7d-4641-b714-19ae0640e13b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.137921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba601a45-2a7d-4641-b714-19ae0640e13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.155816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a7abe-d3c4-48a9-a681-e52c463b2880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ee2a7abe-d3c4-48a9-a681-e52c463b2880 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:03.166688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:03.167670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:03.168465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:03.172688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 182/365] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:03.177311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-ba601a45-2a7d-4641-b714-19ae0640e13b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:03.178982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ba601a45-2a7d-4641-b714-19ae0640e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:03.179394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a7abe-d3c4-48a9-a681-e52c463b2880 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:03.236412 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:03.303735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 184/366] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:00:03.305977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4da1c6-8898-4eac-b918-af117bb7abe2 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.306329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4da1c6-8898-4eac-b918-af117bb7abe2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.306612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4da1c6-8898-4eac-b918-af117bb7abe2 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.308719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 185/367] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:03.314052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d730-7d77-4aa9-af49-faf312c48b84 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.314338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d730-7d77-4aa9-af49-faf312c48b84 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.314617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d730-7d77-4aa9-af49-faf312c48b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.334003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a076d730-7d77-4aa9-af49-faf312c48b84 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:03.335136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a076d730-7d77-4aa9-af49-faf312c48b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:03.346166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a7abe-d3c4-48a9-a681-e52c463b2880 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:03.389965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:03.391113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.392341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.393384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.394427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:03.396042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:03.397630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:03.414672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 183/368] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:03.420684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:03.478515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 186/369] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:03.535545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:03.553804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd7ef6-eace-40a9-9b29-747b98b5032f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:03.598085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e439dd-0778-44de-9c8c-3f1c5da2119e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:03.599106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e439dd-0778-44de-9c8c-3f1c5da2119e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.599611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e439dd-0778-44de-9c8c-3f1c5da2119e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.600097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e439dd-0778-44de-9c8c-3f1c5da2119e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.614333 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e439dd-0778-44de-9c8c-3f1c5da2119e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:03.616087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e439dd-0778-44de-9c8c-3f1c5da2119e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c7e439dd-0778-44de-9c8c-3f1c5da2119e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:03.638248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:03.639205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:03.640064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:03.640064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:03.640064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:03.647561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 184/370] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:03.656227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076d730-7d77-4aa9-af49-faf312c48b84 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:03.665074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:03.681043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4df5-d6f5-4c03-b257-9773010aac57 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.681444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4df5-d6f5-4c03-b257-9773010aac57 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.681755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4df5-d6f5-4c03-b257-9773010aac57 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.683443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 185/371] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:03.688692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4df5-d6f5-4c03-b257-9773010aac57 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:03.801339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076d730-7d77-4aa9-af49-faf312c48b84 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:03.806286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4df5-d6f5-4c03-b257-9773010aac57 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:03.836997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276af41e-d845-433b-bee3-33ef83eb8b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:03.837937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276af41e-d845-433b-bee3-33ef83eb8b26 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.838942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276af41e-d845-433b-bee3-33ef83eb8b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.839577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276af41e-d845-433b-bee3-33ef83eb8b26 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.840437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276af41e-d845-433b-bee3-33ef83eb8b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:03.840741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276af41e-d845-433b-bee3-33ef83eb8b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:03.842193 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276af41e-d845-433b-bee3-33ef83eb8b26 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:03.846102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdec13-98d2-4e85-895b-03e2bb60e0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:03.846730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdec13-98d2-4e85-895b-03e2bb60e0ef None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.847008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdec13-98d2-4e85-895b-03e2bb60e0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.847308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdec13-98d2-4e85-895b-03e2bb60e0ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.858921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 187/372] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:03.858921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ed56a-47d0-4013-8f0f-0bea5876d51c None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:03.858921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ed56a-47d0-4013-8f0f-0bea5876d51c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:03.858921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ed56a-47d0-4013-8f0f-0bea5876d51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:03.863586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdec13-98d2-4e85-895b-03e2bb60e0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-49cdec13-98d2-4e85-895b-03e2bb60e0ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:03.868514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:03.869415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:03.870264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:03.875164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 186/373] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:03.875815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdec13-98d2-4e85-895b-03e2bb60e0ef None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:03.877601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9e3ed56a-47d0-4013-8f0f-0bea5876d51c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:03.878783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9e3ed56a-47d0-4013-8f0f-0bea5876d51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:03.881359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:03.943395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:03.952633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 188/374] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:03.953707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ed56a-47d0-4013-8f0f-0bea5876d51c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.004513 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdec13-98d2-4e85-895b-03e2bb60e0ef None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.048550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067d60f-75e6-4bf8-be41-d92d30e491da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.049664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d60f-75e6-4bf8-be41-d92d30e491da None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.050155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d60f-75e6-4bf8-be41-d92d30e491da None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.050608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067d60f-75e6-4bf8-be41-d92d30e491da None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.068372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067d60f-75e6-4bf8-be41-d92d30e491da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3067d60f-75e6-4bf8-be41-d92d30e491da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:04.074953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:04.075901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:04.076697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:04.083204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 187/375] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:04.084333 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067d60f-75e6-4bf8-be41-d92d30e491da None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.090934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ed56a-47d0-4013-8f0f-0bea5876d51c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.142024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.142976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.143394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.143771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.153365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.155520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:04.172050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:04.173123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:04.174012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:04.178525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 189/376] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 24 13:00:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:04.179403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.181104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:04.221335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067d60f-75e6-4bf8-be41-d92d30e491da None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.243048 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:04.259117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.259753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.260073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.260497 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.261230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:04.261529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.262906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.270736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 188/377] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:04.271614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.294289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bbe8cd-ed19-4225-9605-179accedeaa8 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.328363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.328891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.329172 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.329450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.330374 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:04.330686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.332074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.340178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 190/378] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:04.340958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.368026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b6a7a-7aa1-40a7-a18c-f961c1538348 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.391604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:04.403623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061b51a-610e-4d3c-8c95-eb7d57d1ecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.404228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061b51a-610e-4d3c-8c95-eb7d57d1ecbb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.404506 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061b51a-610e-4d3c-8c95-eb7d57d1ecbb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.404777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061b51a-610e-4d3c-8c95-eb7d57d1ecbb None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.405498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061b51a-610e-4d3c-8c95-eb7d57d1ecbb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:04.405802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061b51a-610e-4d3c-8c95-eb7d57d1ecbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.407251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061b51a-610e-4d3c-8c95-eb7d57d1ecbb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.440941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4e92-e0bf-4421-ab71-5aea9b9b017a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.443169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 189/379] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:04.444384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4103-45b9-4625-8610-bc34591efcc9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.444668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4103-45b9-4625-8610-bc34591efcc9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.444940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a4103-45b9-4625-8610-bc34591efcc9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.446118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 190/380] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:04.446856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4103-45b9-4625-8610-bc34591efcc9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.461712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:04.481513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2dff6-7697-4b14-9da3-9eeb5f0b7d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.482143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2dff6-7697-4b14-9da3-9eeb5f0b7d44 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.482421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2dff6-7697-4b14-9da3-9eeb5f0b7d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.482695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2dff6-7697-4b14-9da3-9eeb5f0b7d44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.492799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2dff6-7697-4b14-9da3-9eeb5f0b7d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-37c2dff6-7697-4b14-9da3-9eeb5f0b7d44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:04.499169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:04.500066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:04.500853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 191/381] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:04.501508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2dff6-7697-4b14-9da3-9eeb5f0b7d44 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.557028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a4103-45b9-4625-8610-bc34591efcc9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.592287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.592841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.593119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.593552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.598740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2dff6-7697-4b14-9da3-9eeb5f0b7d44 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.603184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.605213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:04.619731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:04.620656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:04.621579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:04.621579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:04.621579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:04.621579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 191/382] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:04.621579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.637341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2805f92-ee85-4e7f-b5a9-257c107befd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.638023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805f92-ee85-4e7f-b5a9-257c107befd6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.638307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805f92-ee85-4e7f-b5a9-257c107befd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.638610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2805f92-ee85-4e7f-b5a9-257c107befd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.649124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2805f92-ee85-4e7f-b5a9-257c107befd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.650566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2805f92-ee85-4e7f-b5a9-257c107befd6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.654969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-b2805f92-ee85-4e7f-b5a9-257c107befd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:04.666130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:04.668137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:04.668980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:04.668980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:04.668980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:04.668980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 192/383] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:04.669196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e973c-e0ef-4834-ae95-9f6ddb967b59 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.669573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e973c-e0ef-4834-ae95-9f6ddb967b59 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.669763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e973c-e0ef-4834-ae95-9f6ddb967b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.687738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7c0e973c-e0ef-4834-ae95-9f6ddb967b59 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:04.688938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c0e973c-e0ef-4834-ae95-9f6ddb967b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:04.711185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:04.726132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09005d-0223-411b-9d2a-f49ef5396dc9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.755354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 193/384] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:04.756453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e973c-e0ef-4834-ae95-9f6ddb967b59 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.766531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.767266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.767859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.768141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.780383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.781712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:04.797196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:04.798290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:04.799173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:04.799173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:04.799173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:04.799173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 192/385] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:04.799402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.872735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e973c-e0ef-4834-ae95-9f6ddb967b59 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.887685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:04.903996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be450043-1df1-439e-a5a3-9360189529ce None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:04.910404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.911273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.911561 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.911837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.912495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:04.912801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.914179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.927115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 194/386] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:04.927115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:04.943451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:04.944058 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:04.944338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:04.944608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:04.945250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:04.945552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:04.946924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:04.947858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:04.965124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 193/387] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:04.965124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.033430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebe436-023e-456f-b232-efe5c9a1b3f6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.066984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc95a8-66ec-4133-b6c4-da5c81f2cef5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.067934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.068541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.068822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.069097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.069735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:05.070040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.071500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.086994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 195/388] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:05.087844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.101788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.102392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.102677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.102952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.103622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:05.104476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.105968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.118588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 194/389] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:05.118588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.129510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:05.170879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:05.190526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2807f5-e26f-47fb-8457-486fcb32bdba None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.222459 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1165a5-85c2-4ef7-b5a3-c270aaeb1d7f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.225127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.225760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.226042 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.226318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.236716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.238167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:05.253122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:05.254189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:05.255530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:05.255530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:05.255530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:05.255530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 196/390] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:05.255530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.258057 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.258683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.259279 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.259557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.268052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.269492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:05.282141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:05.283371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 195/391] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:05.284588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.348635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:05.359794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e0db3-2526-44a2-b6f7-8bdc53e34008 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.384834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c3f92-678c-49bc-a4f0-6b7e7cf5709b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.389527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:05.394641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.395297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.395581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.395856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.406255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.407733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.419656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.420275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.420558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.420831 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:05.421564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:05.422480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:05.423340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:05.423340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:05.423340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:05.423340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 197/392] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:05.423574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.434281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.435765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:05.446826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:05.448998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 196/393] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:05.450315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.527853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dee8c3-8b9e-4ea7-b6fc-cb8fb4e2af03 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.548916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1372fd-c1d4-4b73-87c2-20b30a89274f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.561071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.561700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.561987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.562268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.568798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.570258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.575596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:05.582877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:05.583749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:05.584594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 198/394] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:05.585072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.589702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.590316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.590593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.590871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.591547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:05.591854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.593218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.605446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 197/395] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:05.606268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.637872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:05.691262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788f726-fa81-4623-9a05-d1329060ef11 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.706319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361b8f7-c661-4055-a366-c6caabb981f1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.724702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.725311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.725595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.725870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.726599 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:05.726909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.728385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.739858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.740467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.740761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.741038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.741681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:05.741993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.743378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.754662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 198/396] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:05.755494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.765902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 199/397] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:05.766738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.808090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:05.862200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f92453-7dab-4821-8b2c-bbdc873f6172 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.864685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bccdc7-2670-4f37-b387-e931c6e59556 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:05.866175 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:05.897008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.897532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.897813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.898084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.898801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:05.899136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.899484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:05.900095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:05.900380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:05.900588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.900691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:05.901962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:05.902275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:05.903737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:05.918432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 199/398] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:05.919111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:05.927466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 200/399] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:05.927466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.018993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d8e4c-5e4d-411c-a87b-1c76a4071353 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.021458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:06.028917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae1289-058d-45a1-ac54-479c61bb3560 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.053856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.054487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.054768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.055068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.062248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.063734 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.066080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8d739-6dc0-459d-9542-d82eb6c37143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.066694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8d739-6dc0-459d-9542-d82eb6c37143 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.066982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8d739-6dc0-459d-9542-d82eb6c37143 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.067286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8d739-6dc0-459d-9542-d82eb6c37143 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.067961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8d739-6dc0-459d-9542-d82eb6c37143 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.069409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8d739-6dc0-459d-9542-d82eb6c37143 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:06.076285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:06.077166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:06.078056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 200/400] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:06.078604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.093815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 201/401] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:00:05 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:06.094638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8d739-6dc0-459d-9542-d82eb6c37143 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.124095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:06.199181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a396109-ecbf-4fb2-b1a3-7cd0824d10ba None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.221351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8d739-6dc0-459d-9542-d82eb6c37143 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.240672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.241452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.242052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.242341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.255646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.257287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.262605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.263278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.263563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.263840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:06.275129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:06.276605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:06.278792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:06.278792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:06.278792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:06.278967 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.279040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 201/402] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:06.279161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.279229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:06.295941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:06.296912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:06.298522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:06.298522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:06.298522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:06.298522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 202/403] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:06.298522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.300123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:06.373645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:06.388131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d5de9-68f9-4e70-93cf-6ab724ffd0a9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.406973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f7fc3-4797-4c85-a51e-86de6cf2ea10 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.424878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f6356-9e6a-4599-98d0-a9c33fdf70d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.425536 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6356-9e6a-4599-98d0-a9c33fdf70d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.425819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6356-9e6a-4599-98d0-a9c33fdf70d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.426099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6356-9e6a-4599-98d0-a9c33fdf70d3 None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/2dff78b5eadc42608afbf2b14dd07e7a/roles/6598739f382742d7b956c17b01862204` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.441029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.441639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.441925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.442084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f6356-9e6a-4599-98d0-a9c33fdf70d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=2dff78b5eadc42608afbf2b14dd07e7a, role_id=6598739f382742d7b956c17b01862204)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.442253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.442935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:06.443280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.443556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f6356-9e6a-4599-98d0-a9c33fdf70d3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.444727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.467199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 202/404] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:06 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/2dff78b5eadc42608afbf2b14dd07e7a/roles/6598739f382742d7b956c17b01862204 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:06.467199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f6356-9e6a-4599-98d0-a9c33fdf70d3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.467779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 203/405] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:06.469113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.557074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:06.603962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f6356-9e6a-4599-98d0-a9c33fdf70d3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.607579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe574ccc-aea7-40d4-8993-17e70a07daec None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.639900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:06.652228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.653123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.653312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.653430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.653684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.653960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.654247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.654339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:06.654570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.654664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.655200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:06.655506 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.656096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.657001 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.671891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 204/406] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:06.672823 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.673313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 203/407] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:06.678812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.803504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b124a-ce70-47ba-8c67-5a9da1b093e9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.804347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e57ef8-bb63-4a46-b51a-2dd3ff6c8de3 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:06.838847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:06.841133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.841859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.842150 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.842522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.851454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:06.852913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:06.853527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:06.854653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:06.855540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.857083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.863129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:06.864896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:06.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:06.873622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:06.874457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:06.874457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:06.874457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:06.874457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 205/408] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:06.875454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:06.882502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:06.883382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:06.884251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:06.884251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:06.884251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:06.884251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:06.884251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:06.884251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:06.884251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:06.884251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:06.887447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 204/409] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:06.888963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:06.924100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:06.987943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021fea1-8967-4142-bdeb-9d8f95a7ffab None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.014934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc6cea7-ef19-4edd-904a-2df7a60ed0c7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.024444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.025445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.025589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.026338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.034226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.035765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:07.048394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:07.049263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:07.050104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:07.050104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:07.050104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:07.050104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:07.050104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:07.050104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:07.050104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:07.050104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:07.054381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 206/410] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:07.055301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:07.060174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.060791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.061076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.061355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.062005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:07.062311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.063712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.078496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 205/411] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:07.079522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:07.133892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:07.184381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:07.190064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f55ef-3073-4572-a954-3c6bee441d61 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.200331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c130bee-2061-47d9-978b-b4a91fbda073 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.236272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.236892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.237176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.237453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.238239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:07.238558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.240153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.244399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.245258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.245544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.245821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.246519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:07.246828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.248935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.258515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 207/412] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:07.259474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:07.269416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 206/413] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:07.270255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:07.378838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07ae8d-988a-4847-8e1e-37bd8c74f213 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.388157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e77eca5-edaf-4ed3-8b52-ab3c5ed6da18 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.397721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:07.424956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.425461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea758e3-fb27-4e56-b6ff-aa7a4644e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.426038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.426152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea758e3-fb27-4e56-b6ff-aa7a4644e725 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.426397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea758e3-fb27-4e56-b6ff-aa7a4644e725 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.426677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea758e3-fb27-4e56-b6ff-aa7a4644e725 None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/fee3f4d7b69f4851b9160f68a3937703/roles/6598739f382742d7b956c17b01862204` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.426840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.427272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.436524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.438479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.442645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea758e3-fb27-4e56-b6ff-aa7a4644e725 None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=fee3f4d7b69f4851b9160f68a3937703, role_id=6598739f382742d7b956c17b01862204)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.444159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea758e3-fb27-4e56-b6ff-aa7a4644e725 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:07.452588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:07.453671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:07.454698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:07.454698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:07.454698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:07.454698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:07.454698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:07.454698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:07.454698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:07.454698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:07.457415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 208/414] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:07.458600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:07.464060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 207/415] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:07 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/fee3f4d7b69f4851b9160f68a3937703/roles/6598739f382742d7b956c17b01862204 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:07.465014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea758e3-fb27-4e56-b6ff-aa7a4644e725 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:07.497893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:07.641029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea758e3-fb27-4e56-b6ff-aa7a4644e725 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.673092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d56f2-84d1-4f37-8b63-4c49b00eb279 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.684870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.685860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.686927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.687442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.688775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:07.689300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.691584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.708170 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 208/416] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:07.714645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:07.719511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a35469-cd45-4ba1-9c58-551139b36b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.720304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35469-cd45-4ba1-9c58-551139b36b32 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.720688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35469-cd45-4ba1-9c58-551139b36b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.721068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a35469-cd45-4ba1-9c58-551139b36b32 None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/19d0fc260e254d2d80c6927c52cd8a37/roles/6598739f382742d7b956c17b01862204` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.737909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:07.740512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a35469-cd45-4ba1-9c58-551139b36b32 None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=19d0fc260e254d2d80c6927c52cd8a37, role_id=6598739f382742d7b956c17b01862204)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.743090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a35469-cd45-4ba1-9c58-551139b36b32 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.765071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 209/417] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:07 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/19d0fc260e254d2d80c6927c52cd8a37/roles/6598739f382742d7b956c17b01862204 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:07.841220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25329a3b-e41f-49d0-8f48-45af646a4144 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:07.879317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd237bb-f798-482f-8e44-38a1fc3408f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:07.880025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd237bb-f798-482f-8e44-38a1fc3408f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:07.880311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd237bb-f798-482f-8e44-38a1fc3408f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:07.880584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd237bb-f798-482f-8e44-38a1fc3408f2 None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/9c414b04fe0544e3a938661bc957c8bc/roles/6598739f382742d7b956c17b01862204` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:07.898161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd237bb-f798-482f-8e44-38a1fc3408f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=9c414b04fe0544e3a938661bc957c8bc, role_id=6598739f382742d7b956c17b01862204)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:07.899577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd237bb-f798-482f-8e44-38a1fc3408f2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:07.925786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 209/418] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:07 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/9c414b04fe0544e3a938661bc957c8bc/roles/6598739f382742d7b956c17b01862204 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:00:08.494932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640037c7-840c-466f-a409-9d369d35adcb None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:08.495383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640037c7-840c-466f-a409-9d369d35adcb None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:08.495557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640037c7-840c-466f-a409-9d369d35adcb None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:08.497002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 210/419] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:08.502493 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc8162-45d2-4d15-9031-dffbe9b28d98 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:08.502781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc8162-45d2-4d15-9031-dffbe9b28d98 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:08.503072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc8162-45d2-4d15-9031-dffbe9b28d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:08.524371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b7cc8162-45d2-4d15-9031-dffbe9b28d98 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:08.526442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b7cc8162-45d2-4d15-9031-dffbe9b28d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:08.615174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:08.627367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 210/420] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:09.047551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640037c7-840c-466f-a409-9d369d35adcb None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:09.113168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71c7a-37c5-45b2-b53a-1c186863e826 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:09.113706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71c7a-37c5-45b2-b53a-1c186863e826 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:09.113706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de71c7a-37c5-45b2-b53a-1c186863e826 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:09.115216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 211/421] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:09.121794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa9b85-42ed-46dd-9e48-9bda880023fc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:09.122085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa9b85-42ed-46dd-9e48-9bda880023fc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:09.122367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fa9b85-42ed-46dd-9e48-9bda880023fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:09.154432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-74fa9b85-42ed-46dd-9e48-9bda880023fc None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:09.155664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-74fa9b85-42ed-46dd-9e48-9bda880023fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:09.159462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:09.215726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640037c7-840c-466f-a409-9d369d35adcb None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:09.284761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 212/422] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:00:09.294655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13462a78-226c-4fb8-aefc-39653ebf4ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:09.295323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13462a78-226c-4fb8-aefc-39653ebf4ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:09.295607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13462a78-226c-4fb8-aefc-39653ebf4ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:09.296230 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13462a78-226c-4fb8-aefc-39653ebf4ad8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:09.310558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13462a78-226c-4fb8-aefc-39653ebf4ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-13462a78-226c-4fb8-aefc-39653ebf4ad8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:09.315404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:09.316271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:09.317040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:09.321270 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 211/423] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:09.324646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa9b85-42ed-46dd-9e48-9bda880023fc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:09.464092 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fa9b85-42ed-46dd-9e48-9bda880023fc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:09.509038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:09.519366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:09.519982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:09.520266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:09.520544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:09.521219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:09.521529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:09.522917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:09.531606 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 213/424] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:09.540153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13462a78-226c-4fb8-aefc-39653ebf4ad8 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:09.658313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13462a78-226c-4fb8-aefc-39653ebf4ad8 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:09.690642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:09.702394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:09.703242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:09.705127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:09.705468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:09.716433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:09.717888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:09.737963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:09.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:09.739713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:09.739713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:09.739713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:09.739713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 212/425] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:09.744853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:09.786815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:09.903641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13801ae0-8799-40b5-8438-7165e736a85c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:09.943857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0812-4e3d-43ef-9f7b-407d6252b2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:09.945138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0812-4e3d-43ef-9f7b-407d6252b2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:09.945623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0812-4e3d-43ef-9f7b-407d6252b2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:09.946336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f0812-4e3d-43ef-9f7b-407d6252b2e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:09.958226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544f396-c690-4995-801b-f1da0e0bf3b3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:09.961869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920f0812-4e3d-43ef-9f7b-407d6252b2e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-920f0812-4e3d-43ef-9f7b-407d6252b2e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:09.969962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:09.970964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:09.971858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:09.974593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 213/426] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:09.980181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0812-4e3d-43ef-9f7b-407d6252b2e7 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.022334 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.022954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.023298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.023581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.024227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:10.024533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:10.025979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:10.052755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 214/427] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:10.060399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.080433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:10.123261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:10.139987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920f0812-4e3d-43ef-9f7b-407d6252b2e7 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:10.175039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4889d830-a1a1-4a83-9aca-6d919e78faae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.175952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d830-a1a1-4a83-9aca-6d919e78faae None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.176380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d830-a1a1-4a83-9aca-6d919e78faae None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.176799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4889d830-a1a1-4a83-9aca-6d919e78faae None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.177597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889d830-a1a1-4a83-9aca-6d919e78faae None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:10.178044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889d830-a1a1-4a83-9aca-6d919e78faae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:10.179391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756af21-8183-4ffb-9d5c-f927e731ff40 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:10.179795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4889d830-a1a1-4a83-9aca-6d919e78faae None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:10.188477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 214/428] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:10.194463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa941f-4764-4ee8-8996-2452bd00949b None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.194911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa941f-4764-4ee8-8996-2452bd00949b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.195363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa941f-4764-4ee8-8996-2452bd00949b None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.197015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 215/429] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:10.198388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa941f-4764-4ee8-8996-2452bd00949b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.222336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baf45c-e319-4728-8398-a8d8d5d6626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.223092 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf45c-e319-4728-8398-a8d8d5d6626c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.223279 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf45c-e319-4728-8398-a8d8d5d6626c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.223605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baf45c-e319-4728-8398-a8d8d5d6626c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.232114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8baf45c-e319-4728-8398-a8d8d5d6626c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:10.233424 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8baf45c-e319-4728-8398-a8d8d5d6626c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e8baf45c-e319-4728-8398-a8d8d5d6626c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:10.246313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:10.247464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:00:10.248568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:10.252229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 215/430] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:10.253483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09532363-cb9b-464a-b3b1-4c80c2e99e56 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.253779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09532363-cb9b-464a-b3b1-4c80c2e99e56 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.254062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09532363-cb9b-464a-b3b1-4c80c2e99e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.273855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-09532363-cb9b-464a-b3b1-4c80c2e99e56 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:10.274978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-09532363-cb9b-464a-b3b1-4c80c2e99e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:10.311005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa941f-4764-4ee8-8996-2452bd00949b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:10.324648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:10.350667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.351679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.352220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.352644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.353930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 216/431] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:10.359909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09532363-cb9b-464a-b3b1-4c80c2e99e56 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.364529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:10.366219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:10.381656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:10.382595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:10.383486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:10.383486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:10.383486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:10.385754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 216/432] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:10.391357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.433385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:10.495873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b3fe8-96b7-4397-ae24-9a906a3bc38d None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:10.503734 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09532363-cb9b-464a-b3b1-4c80c2e99e56 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:10.531944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a6ce7-e282-4ae5-8034-d3564b382e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.532913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a6ce7-e282-4ae5-8034-d3564b382e22 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.533427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a6ce7-e282-4ae5-8034-d3564b382e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.533898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a6ce7-e282-4ae5-8034-d3564b382e22 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.534753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a6ce7-e282-4ae5-8034-d3564b382e22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:10.535279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a6ce7-e282-4ae5-8034-d3564b382e22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:10.536870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65a6ce7-e282-4ae5-8034-d3564b382e22 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:10.542201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea4fc9-4a88-438c-a30f-57af5b948873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.542861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4fc9-4a88-438c-a30f-57af5b948873 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.543176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4fc9-4a88-438c-a30f-57af5b948873 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.543456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea4fc9-4a88-438c-a30f-57af5b948873 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.544252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea4fc9-4a88-438c-a30f-57af5b948873 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:10.544565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea4fc9-4a88-438c-a30f-57af5b948873 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:10.546034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea4fc9-4a88-438c-a30f-57af5b948873 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:10.555160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 217/433] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:10.556687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b0a1f-6b07-40cc-9068-b16b4d6bad2d None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.557223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b0a1f-6b07-40cc-9068-b16b4d6bad2d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.557696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b0a1f-6b07-40cc-9068-b16b4d6bad2d None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.559485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 218/434] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:10.561771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b0a1f-6b07-40cc-9068-b16b4d6bad2d None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.568833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 217/435] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:10.570145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6073f-4776-4cf3-a5aa-0ff9ac3a430c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.570434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6073f-4776-4cf3-a5aa-0ff9ac3a430c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.570713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6073f-4776-4cf3-a5aa-0ff9ac3a430c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.584919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:10.593627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-16d6073f-4776-4cf3-a5aa-0ff9ac3a430c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:10.595733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-16d6073f-4776-4cf3-a5aa-0ff9ac3a430c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:10.671571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 218/436] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:10.675841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6073f-4776-4cf3-a5aa-0ff9ac3a430c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.678970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b0a1f-6b07-40cc-9068-b16b4d6bad2d None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:10.710831 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:10.717406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.718040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.718325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.718601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.725858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:10.727337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:10.739619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:10.740554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:10.741436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 219/437] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:10.743849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.792159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6073f-4776-4cf3-a5aa-0ff9ac3a430c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:10.826128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dbbff-f5df-486b-9556-71b8f85bf93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.826751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dbbff-f5df-486b-9556-71b8f85bf93c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.827027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dbbff-f5df-486b-9556-71b8f85bf93c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.827326 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dbbff-f5df-486b-9556-71b8f85bf93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.830401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:10.838725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421dbbff-f5df-486b-9556-71b8f85bf93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:10.840295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421dbbff-f5df-486b-9556-71b8f85bf93c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:10.856027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8aad84-cfec-4529-9db3-a67eb22a6457 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-421dbbff-f5df-486b-9556-71b8f85bf93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:10.857296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:10.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:10.859022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:10.859022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:10.859022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:10.859022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 219/438] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:10.859698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f707ae9-f36c-49a2-96f9-0f806e5d5813 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.859887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f707ae9-f36c-49a2-96f9-0f806e5d5813 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.860122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f707ae9-f36c-49a2-96f9-0f806e5d5813 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.861587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 220/439] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:10.862286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f707ae9-f36c-49a2-96f9-0f806e5d5813 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.902501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a02f9-fddc-4b58-b921-2b2981315a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:10.903162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a02f9-fddc-4b58-b921-2b2981315a64 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:10.903449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a02f9-fddc-4b58-b921-2b2981315a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:10.903784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a02f9-fddc-4b58-b921-2b2981315a64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:10.915166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0a02f9-fddc-4b58-b921-2b2981315a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-5a0a02f9-fddc-4b58-b921-2b2981315a64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:10.920926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:10.921976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 220/440] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:10.922894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a02f9-fddc-4b58-b921-2b2981315a64 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:10.953701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:10.976861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f707ae9-f36c-49a2-96f9-0f806e5d5813 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.010858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6e0b2-6dbe-4955-80d4-4987ed156373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.011480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6e0b2-6dbe-4955-80d4-4987ed156373 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.011765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6e0b2-6dbe-4955-80d4-4987ed156373 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.012041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6e0b2-6dbe-4955-80d4-4987ed156373 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.012766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6e0b2-6dbe-4955-80d4-4987ed156373 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:11.013072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6e0b2-6dbe-4955-80d4-4987ed156373 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.014500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6e0b2-6dbe-4955-80d4-4987ed156373 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.026485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a02f9-fddc-4b58-b921-2b2981315a64 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.031120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 221/441] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:11.032302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdf946-aec0-4584-a31e-a4d04422158c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.032592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdf946-aec0-4584-a31e-a4d04422158c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.032872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdf946-aec0-4584-a31e-a4d04422158c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.051110 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-94fdf946-aec0-4584-a31e-a4d04422158c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:11.052220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-94fdf946-aec0-4584-a31e-a4d04422158c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:11.065399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.066012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.066293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.066568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.067258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:11.067540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.068901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.075324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.085309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 221/442] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:11.086109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.124916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 222/443] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:11.139555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdf946-aec0-4584-a31e-a4d04422158c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.197215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.201899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146cd45d-c9d0-498d-9b24-f16693960052 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.233814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdf946-aec0-4584-a31e-a4d04422158c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.237361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.237971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.238255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.238530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.239223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:11.239531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.240918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.253643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 222/444] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:11.253643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.277361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646fe14-c855-4373-8f13-af0afb3a0cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.278002 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646fe14-c855-4373-8f13-af0afb3a0cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.278289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646fe14-c855-4373-8f13-af0afb3a0cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.278571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646fe14-c855-4373-8f13-af0afb3a0cf1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.289805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646fe14-c855-4373-8f13-af0afb3a0cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9646fe14-c855-4373-8f13-af0afb3a0cf1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:11.294697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:11.295779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:11.296701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 223/445] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:11.297532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646fe14-c855-4373-8f13-af0afb3a0cf1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.318366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.363241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d7b30b-ee8c-4e55-a025-3c333308d771 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.401962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ed1c7-b4a8-43de-aca3-7ef88b158efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.402579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ed1c7-b4a8-43de-aca3-7ef88b158efd None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.402867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ed1c7-b4a8-43de-aca3-7ef88b158efd None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.403430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ed1c7-b4a8-43de-aca3-7ef88b158efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.412544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646fe14-c855-4373-8f13-af0afb3a0cf1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.439679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.449368 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.449988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.450265 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.450534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.457219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.458655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:11.470288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:11.471153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:11.472071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 224/446] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:11.472523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.477120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ed1c7-b4a8-43de-aca3-7ef88b158efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.478591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ed1c7-b4a8-43de-aca3-7ef88b158efd None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.540031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.572518 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976dc6a-5678-4e22-8eda-5165c0b9a6b9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.583691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 223/447] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:00:11.584519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ed1c7-b4a8-43de-aca3-7ef88b158efd None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.607545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02228c70-3760-4889-b4af-a110378a299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.608174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228c70-3760-4889-b4af-a110378a299a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.608466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228c70-3760-4889-b4af-a110378a299a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.608747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02228c70-3760-4889-b4af-a110378a299a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.619238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02228c70-3760-4889-b4af-a110378a299a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.621259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02228c70-3760-4889-b4af-a110378a299a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-02228c70-3760-4889-b4af-a110378a299a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:11.636014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:11.636967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:11.638073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:11.638073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:11.638073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:11.638073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 225/448] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:11.638458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02228c70-3760-4889-b4af-a110378a299a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.677306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.689832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ed1c7-b4a8-43de-aca3-7ef88b158efd None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.723201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.723804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.724086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.724359 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.725010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:11.725314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.726682 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.734890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 224/449] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:11.735702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.738079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02228c70-3760-4889-b4af-a110378a299a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.758723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.773417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190da0a3-fb38-48ce-903d-7de6e1240f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.774041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190da0a3-fb38-48ce-903d-7de6e1240f3c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.774325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190da0a3-fb38-48ce-903d-7de6e1240f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.774602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190da0a3-fb38-48ce-903d-7de6e1240f3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.785443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190da0a3-fb38-48ce-903d-7de6e1240f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-190da0a3-fb38-48ce-903d-7de6e1240f3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:11.790153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:11.791233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:11.792174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 226/450] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:11.796061 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190da0a3-fb38-48ce-903d-7de6e1240f3c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.836657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb6307-6874-43b5-8ec6-6bfa2d763535 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.870795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.871426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.871710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.871984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.872702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:11.873036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.874466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.886549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 225/451] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:11.887362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.898959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.902758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190da0a3-fb38-48ce-903d-7de6e1240f3c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:11.937525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:11.938139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:11.938419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:11.938692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:11.949349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:11.950786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:11.965078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:11.966037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:11.966882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:11.966882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:11.966882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:11.966882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 227/452] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:11.967137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:11.973622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:11.994204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff7279-0f67-4e59-a4b7-6c0be1977037 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.028671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.029280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.029563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.029841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.030478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:12.030784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.032213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.043827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 226/453] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:12.044702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.067138 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff494a-922b-4f1e-b308-df95f9895191 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.105322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.105874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.106162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.106440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.116783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.118223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:12.132505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:12.133387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:12.134219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:12.134219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:12.134219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:12.134219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 228/454] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:12.134427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.137416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:12.151949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e763-8066-47cf-bf96-24e0e5070ef4 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.186091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.186695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.186979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.187283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.187941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:12.188248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.189626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.197603 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 227/455] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:12.198406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.200887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:12.238434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdcdd6-7299-4f42-b406-aefc38e073ef None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.272347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36d2d5-2b85-48fd-b87a-1256ab704025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.272990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d2d5-2b85-48fd-b87a-1256ab704025 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.273271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d2d5-2b85-48fd-b87a-1256ab704025 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.273546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d2d5-2b85-48fd-b87a-1256ab704025 None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/5a63131a456548c8a3a3e85dbeba5a48/roles/6598739f382742d7b956c17b01862204` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.288720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36d2d5-2b85-48fd-b87a-1256ab704025 None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=5a63131a456548c8a3a3e85dbeba5a48, role_id=6598739f382742d7b956c17b01862204)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.290192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b36d2d5-2b85-48fd-b87a-1256ab704025 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.300131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad23a2-eaa3-47fd-b33a-7cdc4d0bf775 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.302938 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 229/456] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:12 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/5a63131a456548c8a3a3e85dbeba5a48/roles/6598739f382742d7b956c17b01862204 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:12.303769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36d2d5-2b85-48fd-b87a-1256ab704025 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.340013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.340656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.340944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.341235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.341884 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:12.342194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.343703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.363187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 228/457] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:12.364142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.365311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:12.421030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b36d2d5-2b85-48fd-b87a-1256ab704025 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.435027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:12.460684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.461332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.461618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.461895 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.470621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.472208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.480662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126240b-0865-49c7-9d9e-b86cf7109ebe None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:12.485043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:12.486137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 230/458] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:12.487226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.522007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.522722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.523095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.523445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.524160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:12.524536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.526547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.542331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 229/459] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:12.543672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.594821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2556c3-a2a3-45dc-aa98-6a3150a798ec None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.608469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:12.631723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.632281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.632567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.632851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.644636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.645287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef64b26-e9d7-4a09-af01-f71b77500608 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.646110 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:12.661191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:12.662151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:12.663000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:12.663000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:12.663000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:12.667354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 231/460] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:12.668528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.671986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:12.686770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.687453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.687747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.688028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.695361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.696891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:12.708342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:12.709395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:12.710435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:12.710435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:12.710435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:12.710435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:12.710435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:12.710435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:12.710435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:12.710435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:12.716673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 230/461] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:12.716673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.785830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5924f1-9c2a-4f90-953b-66b431910533 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.821183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.821790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.822074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.822350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.823529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:12.824049 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.825667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.847102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 232/462] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:12.847102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.852311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4976b84-5700-47a1-be0d-f3c8718bed7f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.872672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:12.894189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.895224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.895620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.896177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.904081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.905642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:12.917049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:12.918163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 24 13:00:12.919356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:12.925760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 231/463] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:12.925760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:12.958276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e10e1b-38bf-4abe-8eec-94fa0ed6e452 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:12.972252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:12.995615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:12.996233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:12.996518 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:12.996797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:12.997445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:12.997753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:12.999163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.015476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 233/464] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:13.015476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.042808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b1e1-7d3d-4f2e-b4df-c690c6736404 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:13.084306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:13.084995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:13.085351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:13.085696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:13.086458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:13.086841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:13.088854 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.101639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 232/465] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:13.102461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.111635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:13.132304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d016b1-83d4-4a3e-b2a1-c1e2d092a720 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:13.167756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663ba1d-c19f-4729-8732-fd84f994dd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:13.168373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663ba1d-c19f-4729-8732-fd84f994dd79 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:13.168656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663ba1d-c19f-4729-8732-fd84f994dd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:13.168934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663ba1d-c19f-4729-8732-fd84f994dd79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:13.216474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e3a2f-3c91-4a31-8d87-642c23372569 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:13.218100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:13.241463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663ba1d-c19f-4729-8732-fd84f994dd79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:13.243085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663ba1d-c19f-4729-8732-fd84f994dd79 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.257201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:13.257809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:13.258091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:13.258365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:13.259127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:13.259429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:13.260971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.283509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 233/466] 10.222.0.42 () {56 vars in 1137 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:13.283509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.328285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:13.349982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 234/467] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:00:13.350954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663ba1d-c19f-4729-8732-fd84f994dd79 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.391904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7a4af-a767-4d9e-b831-bea7a55cb5a4 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:13.427425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:13.427980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:13.428261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:13.428533 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:13.436502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:13.437947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:13.450199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:13.451128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 234/468] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:13.452063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.464629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663ba1d-c19f-4729-8732-fd84f994dd79 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:13.477728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:13.500473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:13.501100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:13.501362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:13.501636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:13.502366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] RBAC: Adding query filter params (role.id=6598739f382742d7b956c17b01862204, scope.project.id=00530732ebe84dababafb9f399452940, user.id=2dff78b5eadc42608afbf2b14dd07e7a) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:13.502675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:13.504173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.510466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 235/469] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/role_assignments?user.id=2dff78b5eadc42608afbf2b14dd07e7a&scope.project.id=00530732ebe84dababafb9f399452940&role.id=6598739f382742d7b956c17b01862204 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:13.511246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.577956 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:13.579710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c61b4b-d97a-4682-bfe3-9ffe05f124ae None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:13.621553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa79c02-f626-4225-994b-33e93d3fea96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:13.622364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa79c02-f626-4225-994b-33e93d3fea96 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:13.623090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa79c02-f626-4225-994b-33e93d3fea96 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:13.623090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa79c02-f626-4225-994b-33e93d3fea96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:13.656308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f70a21-268a-4dd5-8068-912a02fa1749 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:13.702783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:13.703385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:13.703665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:13.703941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:13.705457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa79c02-f626-4225-994b-33e93d3fea96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:13.706977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa79c02-f626-4225-994b-33e93d3fea96 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.722137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:13.724122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.735846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:13.740732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:13.741620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:13.742479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:13.742479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:13.742479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:13.742479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 236/470] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:13.742479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.844331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 235/471] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:00:13.845813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa79c02-f626-4225-994b-33e93d3fea96 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.866998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:13.871818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89834f4-5d50-4152-9e29-81cb3dfdf997 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:13.909821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:13.910736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:13.911194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:13.911612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:13.912488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:13.912936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:13.914528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:13.933645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 237/472] 10.222.0.42 () {56 vars in 1139 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:13.934772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:13.989084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa79c02-f626-4225-994b-33e93d3fea96 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.016234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:14.032849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.033686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.034111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.034524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.035337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:14.035808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.037335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.049738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 236/473] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:14.050932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.062587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc6017-b4d6-4b09-90b1-54bea57f254d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.109555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.110510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.110977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.111499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.123086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.124817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:14.141332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:14.142495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:14.143426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:14.143426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:14.143426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:14.146801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 238/474] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:14.147867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.162230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:14.180484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87f8d9-ddf9-4abf-9e8f-1c5f7c1f990c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.218770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fee743-924f-4301-804c-b7e0a9861f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.220076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fee743-924f-4301-804c-b7e0a9861f17 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.220533 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fee743-924f-4301-804c-b7e0a9861f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.220953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fee743-924f-4301-804c-b7e0a9861f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.276294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:14.280807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fc9ed-ac20-4467-838e-6a013f1c1c5e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.304769 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fee743-924f-4301-804c-b7e0a9861f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.307422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fee743-924f-4301-804c-b7e0a9861f17 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.320425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.321289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.321704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.322122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.330094 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.331777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:14.343759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:14.344815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:14.345856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:14.345856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:14.345856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:14.345856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:14.345856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:14.345856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:14.345856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:14.345856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:14.347899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 239/475] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:14.348831 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.426344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:14.445086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 237/476] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:00:14.446350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fee743-924f-4301-804c-b7e0a9861f17 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.464601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf12b9f-5f83-4014-a5b5-5e9f216046ec None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.501929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.502845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.503317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.503762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.504586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:14.505045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.507011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.520612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 240/477] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:14.521774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.547961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:14.568310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fee743-924f-4301-804c-b7e0a9861f17 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.603457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7651148f-6229-4727-93ce-43cf7dc5d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.604738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651148f-6229-4727-93ce-43cf7dc5d0dd None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.605026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651148f-6229-4727-93ce-43cf7dc5d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.605301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7651148f-6229-4727-93ce-43cf7dc5d0dd None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.605984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651148f-6229-4727-93ce-43cf7dc5d0dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:14.606293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651148f-6229-4727-93ce-43cf7dc5d0dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.607763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7651148f-6229-4727-93ce-43cf7dc5d0dd None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.623926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 238/478] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:14.623926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d47c-5fdf-4c7e-bfd7-e0139ee24ced None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.623926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d47c-5fdf-4c7e-bfd7-e0139ee24ced None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.623926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d47c-5fdf-4c7e-bfd7-e0139ee24ced None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.623926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 239/479] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:14.623926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106d47c-5fdf-4c7e-bfd7-e0139ee24ced None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.635947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f286869-0f8c-4566-824e-f5eb325cfce4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.664204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:14.676920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.677619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.677817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.678088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.689714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.691127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:14.707027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:14.707951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:14.708960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:14.708960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:14.708960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:14.708960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 241/480] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:14.708960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.738825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106d47c-5fdf-4c7e-bfd7-e0139ee24ced None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.774712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc6c58-b864-4b81-9bad-b544cb6c94ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.775396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc6c58-b864-4b81-9bad-b544cb6c94ba None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.776305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc6c58-b864-4b81-9bad-b544cb6c94ba None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.776587 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc6c58-b864-4b81-9bad-b544cb6c94ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.783413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc6c58-b864-4b81-9bad-b544cb6c94ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.785476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc6c58-b864-4b81-9bad-b544cb6c94ba None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.787765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-21bc6c58-b864-4b81-9bad-b544cb6c94ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:14.799736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:14.800566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:14.801415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 240/481] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:14.802566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73632a-080f-49cb-a1cc-833ddc7a2c1b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.802566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73632a-080f-49cb-a1cc-833ddc7a2c1b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.802566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73632a-080f-49cb-a1cc-833ddc7a2c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.820918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-bd73632a-080f-49cb-a1cc-833ddc7a2c1b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:14.821288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ae9afc-adce-4247-8f5b-f0553c99d313 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:14.822174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-bd73632a-080f-49cb-a1cc-833ddc7a2c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:14.858135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:14.858745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:14.859085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:14.859324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:14.860048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] RBAC: Adding query filter params (role.id=6598739f382742d7b956c17b01862204, scope.project.id=00530732ebe84dababafb9f399452940, user.id=fee3f4d7b69f4851b9160f68a3937703) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:14.860361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:14.861807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:14.867630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 242/482] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/role_assignments?user.id=fee3f4d7b69f4851b9160f68a3937703&scope.project.id=00530732ebe84dababafb9f399452940&role.id=6598739f382742d7b956c17b01862204 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:14.868426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.919235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 241/483] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:14.920482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73632a-080f-49cb-a1cc-833ddc7a2c1b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:14.947350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:15.066654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5be0d5-5b06-4117-b7c0-e9aa6a11e64a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:15.111639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73632a-080f-49cb-a1cc-833ddc7a2c1b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:15.113479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:15.114260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.115659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.116093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.116893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:15.117350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:15.118877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:15.137034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 243/484] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:15.142186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:15.154571 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:15.161701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:15.162320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.162605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.162886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.163621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:15.164760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:15.165591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:15.181354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 242/485] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:15.188834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:15.283776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4b7dd-823c-49be-8243-d8ebbaf9bd70 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:15.287340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:15.330152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf7d5d0-b085-4781-b245-ab72a0f71c7c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:15.334554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:15.335423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.335855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.336278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.346053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:15.348066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:15.368133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:15.369722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:15.370878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:15.370878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:15.370878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:15.370878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:15.370878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:15.370878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:15.370878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:15.370878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:15.376955 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 244/486] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:15.376955 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:15.379685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:15.380419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.380800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.381450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.382312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] RBAC: Adding query filter params (role.id=6598739f382742d7b956c17b01862204, scope.project.id=00530732ebe84dababafb9f399452940, user.id=19d0fc260e254d2d80c6927c52cd8a37) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:15.382720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:15.385732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:15.393139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 243/487] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:15 2022] GET /identity/v3/role_assignments?user.id=19d0fc260e254d2d80c6927c52cd8a37&scope.project.id=00530732ebe84dababafb9f399452940&role.id=6598739f382742d7b956c17b01862204 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:15.394142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:15.517455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:15.583476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da86b8-f1cc-405c-921b-a9bcfb91e952 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:15.584717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0126f-33a3-4f43-bdb8-af554187fedb None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:15.618753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98652c14-12ac-47b0-8948-fb4febaa4686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:15.619599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652c14-12ac-47b0-8948-fb4febaa4686 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.620256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652c14-12ac-47b0-8948-fb4febaa4686 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.620692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98652c14-12ac-47b0-8948-fb4febaa4686 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.628502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:15.629305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.629759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.630181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.631292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:15.631756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:15.632391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98652c14-12ac-47b0-8948-fb4febaa4686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:15.633596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-98652c14-12ac-47b0-8948-fb4febaa4686 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:15.638524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:15.639438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:15.640510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 245/488] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:15.645373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98652c14-12ac-47b0-8948-fb4febaa4686 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:15.653542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 244/489] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:15.664372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:15.687510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:15.790470 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d3d374-343e-4b40-82a0-5c5a997ce5f7 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:15.792180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98652c14-12ac-47b0-8948-fb4febaa4686 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:15.811426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:15.835113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:15.835767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.836081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.836426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.837270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:15.837595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:15.839734 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:15.840144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4434c9-e1ae-4b78-881e-98f4ad447064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:15.840980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4434c9-e1ae-4b78-881e-98f4ad447064 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.841744 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4434c9-e1ae-4b78-881e-98f4ad447064 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.842676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4434c9-e1ae-4b78-881e-98f4ad447064 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.843624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4434c9-e1ae-4b78-881e-98f4ad447064 None admin] RBAC: Adding query filter params (role.id=6598739f382742d7b956c17b01862204, scope.project.id=00530732ebe84dababafb9f399452940, user.id=9c414b04fe0544e3a938661bc957c8bc) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:15.844107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4434c9-e1ae-4b78-881e-98f4ad447064 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:15.846256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4434c9-e1ae-4b78-881e-98f4ad447064 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:15.851199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 246/490] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:15.860786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 245/491] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:15 2022] GET /identity/v3/role_assignments?user.id=9c414b04fe0544e3a938661bc957c8bc&scope.project.id=00530732ebe84dababafb9f399452940&role.id=6598739f382742d7b956c17b01862204 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:15.862693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:15.869121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a59e3-c835-428a-8a59-120c09f521fb None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.870278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a59e3-c835-428a-8a59-120c09f521fb None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.870727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a59e3-c835-428a-8a59-120c09f521fb None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.872383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 246/492] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:15.880102 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561d113-5cf3-401d-9d4b-b356d40c3c10 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:15.880653 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561d113-5cf3-401d-9d4b-b356d40c3c10 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:15.881350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561d113-5cf3-401d-9d4b-b356d40c3c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:15.907870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0561d113-5cf3-401d-9d4b-b356d40c3c10 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:15.909600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0561d113-5cf3-401d-9d4b-b356d40c3c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:15.983520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:16.011597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d949f88f-c317-44ea-a09e-cb5ea186eb91 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:16.037041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 247/493] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:16.084070 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:16.084738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:16.085025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:16.085353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:16.107302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:16.108811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:16.130577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:16.132355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:16.133357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:16.133357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:16.133357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:16.133357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 247/494] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:16.138716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561d113-5cf3-401d-9d4b-b356d40c3c10 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:16.254782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561d113-5cf3-401d-9d4b-b356d40c3c10 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:16.265384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:16.292426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:16.293045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:16.293330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:16.293608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:16.294260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:16.294566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:16.295975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:16.309074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 248/495] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:16.316211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:16.431429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:16.432581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04c889-d8ad-4fe1-90a3-de41b03553aa None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:16.497439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:16.498080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:16.498363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:16.498636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:16.511711 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:16.513205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:16.517455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:16.536108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:16.537064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:00:16.538135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:16.538655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 248/496] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:00:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:16.543408 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:16.599845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b14104-b4fa-4725-a750-a2b4f0322a4f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:16.637740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79772e2c-029b-4bb7-997c-d5a4a6fa4349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:16.643077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772e2c-029b-4bb7-997c-d5a4a6fa4349 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:16.643077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772e2c-029b-4bb7-997c-d5a4a6fa4349 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:16.643077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772e2c-029b-4bb7-997c-d5a4a6fa4349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:16.649634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79772e2c-029b-4bb7-997c-d5a4a6fa4349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-79772e2c-029b-4bb7-997c-d5a4a6fa4349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:16.655341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:16.656248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:16.657080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 249/497] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:00:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:16.665774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79772e2c-029b-4bb7-997c-d5a4a6fa4349 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:16.699461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:16.778555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2b5f8-2f4d-4782-9833-4f75e671a118 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:16.791621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79772e2c-029b-4bb7-997c-d5a4a6fa4349 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:16.819097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:16.819097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:16.819097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:16.819097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:16.819501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:16.819501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:16.819950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:16.845363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 249/498] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 24 13:00:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:16.856059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:16.872897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:16.873513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:16.873796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:16.874074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:16.874798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:16.875131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:16.876608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:16.890822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 250/499] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:16.897756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:16.979821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:17.008743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ea56b-306b-42c9-91c6-1e2e3d1ffe8c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:17.073493 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66869c-f314-4454-9005-792b3f42336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:17.074280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66869c-f314-4454-9005-792b3f42336e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:17.074564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66869c-f314-4454-9005-792b3f42336e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:17.074844 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66869c-f314-4454-9005-792b3f42336e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:17.095617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:17.112110 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def25e8-47a4-489e-84e5-e427c3aaf252 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:17.162778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:17.163452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:17.163738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:17.164013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:17.187584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:17.189063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:17.192663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66869c-f314-4454-9005-792b3f42336e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:17.194120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66869c-f314-4454-9005-792b3f42336e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:17.209521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:17.210981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:17.211873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:17.211873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:17.211873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:17.215225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 251/500] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:17.216568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:17.327527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f373fd-fb14-460f-8d79-7b4094f86d42 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:17.329749 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:17.364386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:17.364995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:17.365280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:17.365559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:17.366204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:17.366510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:17.368209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:17.385555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 252/501] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:17.387452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:17.390120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:17.416776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 250/502] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 24 13:00:17.423032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66869c-f314-4454-9005-792b3f42336e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:17.511759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292a8bd-b23d-42a0-8e7a-3880b5b79f65 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:17.565582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639621b-76bb-48cf-9b1f-0b52c1dc6373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:17.566236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639621b-76bb-48cf-9b1f-0b52c1dc6373 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:17.566517 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639621b-76bb-48cf-9b1f-0b52c1dc6373 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:17.566794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639621b-76bb-48cf-9b1f-0b52c1dc6373 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:17.586492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639621b-76bb-48cf-9b1f-0b52c1dc6373 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:17.588430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639621b-76bb-48cf-9b1f-0b52c1dc6373 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-3639621b-76bb-48cf-9b1f-0b52c1dc6373 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:17.609893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:17.610960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:17.611925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:17.615236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 253/503] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 24 13:00:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:17.616983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31640-654a-4466-96de-76ff1f901890 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:17.617273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31640-654a-4466-96de-76ff1f901890 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:17.617550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea31640-654a-4466-96de-76ff1f901890 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:17.618832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 254/504] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:17.620819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea31640-654a-4466-96de-76ff1f901890 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:17.638234 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66869c-f314-4454-9005-792b3f42336e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:17.676724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb22fa-b757-4d52-9d70-e8db09e32d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:17.677720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb22fa-b757-4d52-9d70-e8db09e32d46 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:17.678004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb22fa-b757-4d52-9d70-e8db09e32d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:17.678278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb22fa-b757-4d52-9d70-e8db09e32d46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:17.690541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb22fa-b757-4d52-9d70-e8db09e32d46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:17.692047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb22fa-b757-4d52-9d70-e8db09e32d46 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:17.704700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e7cb22fa-b757-4d52-9d70-e8db09e32d46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:17.709903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:17.710952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:17.712502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:17.712502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:17.712502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:17.712502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 251/505] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:17.712502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e43a-5818-4e64-8f18-c1599aac560a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:17.712789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e43a-5818-4e64-8f18-c1599aac560a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:17.713259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e43a-5818-4e64-8f18-c1599aac560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:17.734405 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9973e43a-5818-4e64-8f18-c1599aac560a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:17.735625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9973e43a-5818-4e64-8f18-c1599aac560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:17.737926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:17.755570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea31640-654a-4466-96de-76ff1f901890 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:17.792885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:17.793518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:17.793800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:17.794075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:17.794808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:17.795153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:17.796864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:17.820767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 255/506] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 24 13:00:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:17.820767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:17.837727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 252/507] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:17.839443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e43a-5818-4e64-8f18-c1599aac560a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:17.956358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf46a9-43fc-4f7a-b590-a624888200e4 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:17.959912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e43a-5818-4e64-8f18-c1599aac560a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:17.997835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:18.015640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.016988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.017478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.017977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.018857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:18.019439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.021053 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.035707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 256/508] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:18.042138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.055824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.056479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.057349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.057627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.083366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:18.104747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.106137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:18.133970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:18.135626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:18.136528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:18.136528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:18.136528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:18.139216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 253/509] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:18.147784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.204760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f875db-f6aa-4f18-bdc4-c8290076bbe2 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.239923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.240586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.240867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.241140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.248576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.250075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:18.262244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:18.263187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:18.264127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 257/510] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:18.264597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.276765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65335d-fec4-4c16-86f0-d70c92df4b06 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.308698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:18.316374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460407-35c8-4ece-884a-569087af4870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.317100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460407-35c8-4ece-884a-569087af4870 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.317383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460407-35c8-4ece-884a-569087af4870 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.317660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460407-35c8-4ece-884a-569087af4870 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.318303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460407-35c8-4ece-884a-569087af4870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:18.318606 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460407-35c8-4ece-884a-569087af4870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.320112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460407-35c8-4ece-884a-569087af4870 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.332529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 254/511] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:18.334448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad3ce6-0cdb-4ddf-a4d6-fea468dc8cec None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.334921 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad3ce6-0cdb-4ddf-a4d6-fea468dc8cec None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.335398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad3ce6-0cdb-4ddf-a4d6-fea468dc8cec None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.337067 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 255/512] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:18.338489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad3ce6-0cdb-4ddf-a4d6-fea468dc8cec None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.374662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d3d00-6972-48cf-a29a-bad9f7553dba None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.393564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:18.411355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.411992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.412276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.412553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.413132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.414497 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:18.426180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:18.427318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:18.428165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 258/513] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:18.429032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.457593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad3ce6-0cdb-4ddf-a4d6-fea468dc8cec None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.495119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3876-71b9-4788-8109-6b0abaa986a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.495727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3876-71b9-4788-8109-6b0abaa986a4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.496014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3876-71b9-4788-8109-6b0abaa986a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.496287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3876-71b9-4788-8109-6b0abaa986a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.496970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3876-71b9-4788-8109-6b0abaa986a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:18.497284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3876-71b9-4788-8109-6b0abaa986a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.498715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3876-71b9-4788-8109-6b0abaa986a4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.511346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 256/514] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:18.512583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5c877-c542-4f3d-9bc8-8368cf1d064d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.512868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5c877-c542-4f3d-9bc8-8368cf1d064d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.513144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5c877-c542-4f3d-9bc8-8368cf1d064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.531091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-40d5c877-c542-4f3d-9bc8-8368cf1d064d None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:18.532339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-40d5c877-c542-4f3d-9bc8-8368cf1d064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:18.542002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74cf05-2063-4b76-872f-f605cee9e963 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.544404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:18.579357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.580061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.580399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.580743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.587462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.589472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:18.600758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:18.601647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:18.602530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 259/515] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:18.603025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.610205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 257/516] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:18.616099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5c877-c542-4f3d-9bc8-8368cf1d064d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.675231 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:18.743217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a4ceb-dfab-47ff-a052-957b8dcd8b57 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.755349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5c877-c542-4f3d-9bc8-8368cf1d064d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.779291 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.779904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.780191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.780467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.781092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:18.781402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.782757 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.787121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 260/517] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:18.791710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.796884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebd2be2-90a9-4af2-aa0a-7a42d6db1c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.797499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd2be2-90a9-4af2-aa0a-7a42d6db1c86 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.797784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd2be2-90a9-4af2-aa0a-7a42d6db1c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.798060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebd2be2-90a9-4af2-aa0a-7a42d6db1c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.798826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd2be2-90a9-4af2-aa0a-7a42d6db1c86 None admin] RBAC: Adding query filter params (role.id=6598739f382742d7b956c17b01862204, scope.project.id=00530732ebe84dababafb9f399452940, user.id=5a63131a456548c8a3a3e85dbeba5a48) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:18.799179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd2be2-90a9-4af2-aa0a-7a42d6db1c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.800614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebd2be2-90a9-4af2-aa0a-7a42d6db1c86 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.807161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 258/518] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/role_assignments?user.id=5a63131a456548c8a3a3e85dbeba5a48&scope.project.id=00530732ebe84dababafb9f399452940&role.id=6598739f382742d7b956c17b01862204 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:18.808470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0a399-9123-4b77-97d3-d59ccd7ed63f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.808761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0a399-9123-4b77-97d3-d59ccd7ed63f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.809036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b0a399-9123-4b77-97d3-d59ccd7ed63f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.810396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 259/519] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:18.815492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0a399-9123-4b77-97d3-d59ccd7ed63f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:18.836896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:18.925945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a03af4-9a86-4bb9-adc0-8d24f4c6934e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.957800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b0a399-9123-4b77-97d3-d59ccd7ed63f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:18.960431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:18.962395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d5e09-08f9-4e51-b965-9e6d7abfbe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:18.963929 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5e09-08f9-4e51-b965-9e6d7abfbe03 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.964334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5e09-08f9-4e51-b965-9e6d7abfbe03 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.964670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5e09-08f9-4e51-b965-9e6d7abfbe03 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:18.965372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d5e09-08f9-4e51-b965-9e6d7abfbe03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:18.965692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d5e09-08f9-4e51-b965-9e6d7abfbe03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:18.967240 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d5e09-08f9-4e51-b965-9e6d7abfbe03 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:18.987019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 261/520] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:18.987019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dfda2-3964-4eeb-903f-1e77a6567456 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:18.987019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dfda2-3964-4eeb-903f-1e77a6567456 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:18.987019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dfda2-3964-4eeb-903f-1e77a6567456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:19.004754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:19.005894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-599dfda2-3964-4eeb-903f-1e77a6567456 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:19.006258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:19.006258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:19.006258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:19.006744 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-599dfda2-3964-4eeb-903f-1e77a6567456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:19.006847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:19.007080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:19.008444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:19.016086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 260/521] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 24 13:00:18 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:19.016830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:19.094333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 262/522] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:19.095326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dfda2-3964-4eeb-903f-1e77a6567456 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:19.116448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:19.170387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9c5b6-065d-4132-83db-706e1ef506b2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:19.216429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:19.217753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:19.218242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:19.219222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:19.221924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dfda2-3964-4eeb-903f-1e77a6567456 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:19.233102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:19.235818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:19.257106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:19.258101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:19.258971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:19.258971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:19.258971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:19.260525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 261/523] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:19.266774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:19.277765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739dba1-8829-40b4-b7d2-e8eaf3293b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:19.278765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739dba1-8829-40b4-b7d2-e8eaf3293b97 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:19.279315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739dba1-8829-40b4-b7d2-e8eaf3293b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:19.279796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739dba1-8829-40b4-b7d2-e8eaf3293b97 None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/2dff78b5eadc42608afbf2b14dd07e7a/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:19.282243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739dba1-8829-40b4-b7d2-e8eaf3293b97 None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=2dff78b5eadc42608afbf2b14dd07e7a, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:19.284030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739dba1-8829-40b4-b7d2-e8eaf3293b97 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:19.294652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:19.303652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 263/524] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:19 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/2dff78b5eadc42608afbf2b14dd07e7a/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:19.390422 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1deae1-2fff-4e0c-b1a0-f51dd569b7f1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:19.433363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739dba1-8829-40b4-b7d2-e8eaf3293b97 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:19.435679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:19.436304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:19.436610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:19.436864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:19.437527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:19.437837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:19.439270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:19.453145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 262/525] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:19.460488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:19.585281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:19.586574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:19.617554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84569910-711e-44e0-8b03-92f8ec6c46b4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:19.628550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739dba1-8829-40b4-b7d2-e8eaf3293b97 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:19.673062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:19.673062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:19.673062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:19.673062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:19.673445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:19.673445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:19.676715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1bf06-088f-4e2d-91f9-7c0e3029ef83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:19.677610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1bf06-088f-4e2d-91f9-7c0e3029ef83 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:19.678094 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1bf06-088f-4e2d-91f9-7c0e3029ef83 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:19.678569 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1bf06-088f-4e2d-91f9-7c0e3029ef83 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:19.679481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1bf06-088f-4e2d-91f9-7c0e3029ef83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:19.681037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:19.682696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:19.683617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 264/526] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:19 2022] GET /identity/v3/services/network => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:19.691314 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1bf06-088f-4e2d-91f9-7c0e3029ef83 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:19.691468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:19.732931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 263/527] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:00:19 2022] POST /identity/v3/users => generated 254 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:00:19.832125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ac247-f0f0-40e8-bf68-0c421a5274c9 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:19.870265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c8e73-56c6-4992-bbc6-69dc1cadbd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:19.870818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c8e73-56c6-4992-bbc6-69dc1cadbd44 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:19.871164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c8e73-56c6-4992-bbc6-69dc1cadbd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:19.871444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c8e73-56c6-4992-bbc6-69dc1cadbd44 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:19.872098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c8e73-56c6-4992-bbc6-69dc1cadbd44 None admin] RBAC: Adding query filter params (name=network) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:19.872462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c8e73-56c6-4992-bbc6-69dc1cadbd44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:19.873907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c8e73-56c6-4992-bbc6-69dc1cadbd44 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:19.879807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 265/528] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:19.888222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1bf06-088f-4e2d-91f9-7c0e3029ef83 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:20.013493 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1bf06-088f-4e2d-91f9-7c0e3029ef83 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:20.059944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26023ab-7387-4999-aaa1-1864a9fb6b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:20.060566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26023ab-7387-4999-aaa1-1864a9fb6b6e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:20.060846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26023ab-7387-4999-aaa1-1864a9fb6b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:20.061115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26023ab-7387-4999-aaa1-1864a9fb6b6e None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:20.061765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26023ab-7387-4999-aaa1-1864a9fb6b6e None admin] RBAC: Adding query filter params (type=network) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:20.062061 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26023ab-7387-4999-aaa1-1864a9fb6b6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:20.063526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26023ab-7387-4999-aaa1-1864a9fb6b6e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:20.069842 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:20.073912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 264/529] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:21.581990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744308b5-bcb5-4002-ad55-c9775133b4a0 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:21.583030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744308b5-bcb5-4002-ad55-c9775133b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:21.583542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744308b5-bcb5-4002-ad55-c9775133b4a0 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:21.585319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 266/530] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:21.591628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde1287-4936-4777-9aa8-02bb180d5f2f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:21.592484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde1287-4936-4777-9aa8-02bb180d5f2f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:21.592962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde1287-4936-4777-9aa8-02bb180d5f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:21.620664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cfde1287-4936-4777-9aa8-02bb180d5f2f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:21.622217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cfde1287-4936-4777-9aa8-02bb180d5f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:21.717331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 265/531] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:21.962195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46273f0-68a6-443e-bbd5-a728627743b3 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:21.962914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46273f0-68a6-443e-bbd5-a728627743b3 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:21.963425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46273f0-68a6-443e-bbd5-a728627743b3 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:21.965216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 267/532] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:21.970523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd37aed-f180-4d80-98a4-c3b967b3bfce None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:21.971111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd37aed-f180-4d80-98a4-c3b967b3bfce None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:21.971595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd37aed-f180-4d80-98a4-c3b967b3bfce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:21.995979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7fd37aed-f180-4d80-98a4-c3b967b3bfce None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:21.997478 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7fd37aed-f180-4d80-98a4-c3b967b3bfce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:22.063508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46273f0-68a6-443e-bbd5-a728627743b3 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:22.075038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:22.090680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 266/533] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:22.092383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44646d70-a74a-4379-8505-00fcb5908eeb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.092914 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44646d70-a74a-4379-8505-00fcb5908eeb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.093397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44646d70-a74a-4379-8505-00fcb5908eeb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.099107 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 267/534] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:22.100842 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154d906-b3e8-420f-836a-285d17044766 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.101342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154d906-b3e8-420f-836a-285d17044766 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.101854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154d906-b3e8-420f-836a-285d17044766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.120855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e154d906-b3e8-420f-836a-285d17044766 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:22.122398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e154d906-b3e8-420f-836a-285d17044766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:22.186301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:22.207172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 268/535] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 24 13:00:22.236766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46273f0-68a6-443e-bbd5-a728627743b3 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:22.299759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63efc0-958a-4f8e-9c75-26ef240ba3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:22.300402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63efc0-958a-4f8e-9c75-26ef240ba3dc None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.300741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63efc0-958a-4f8e-9c75-26ef240ba3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.301007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63efc0-958a-4f8e-9c75-26ef240ba3dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.317192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63efc0-958a-4f8e-9c75-26ef240ba3dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-2e63efc0-958a-4f8e-9c75-26ef240ba3dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:22.322010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:22.322992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:22.324476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 268/536] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:00:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:22.329120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154d906-b3e8-420f-836a-285d17044766 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:22.441821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154d906-b3e8-420f-836a-285d17044766 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:22.478346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:22.479447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.479945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.480442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.481338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:22.481846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:22.483458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:22.488080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63efc0-958a-4f8e-9c75-26ef240ba3dc None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:22.493478 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 269/537] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:22.503516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:22.566574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:22.611589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:22.620504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553e8d1-2787-4066-bbb3-c9a18cfa731c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:22.652348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63efc0-958a-4f8e-9c75-26ef240ba3dc None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:22.657667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:22.658307 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.658593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.658872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.672142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:22.673627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:22.689118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9779ad-17d2-4576-8fe3-ae0cf78eaec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:22.689340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:22.690228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:22.691136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:22.691136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:22.691136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:22.691307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9779ad-17d2-4576-8fe3-ae0cf78eaec3 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.691307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9779ad-17d2-4576-8fe3-ae0cf78eaec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.691307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9779ad-17d2-4576-8fe3-ae0cf78eaec3 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.691583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 270/538] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:22.691652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:22.695849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9779ad-17d2-4576-8fe3-ae0cf78eaec3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:22.697849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9779ad-17d2-4576-8fe3-ae0cf78eaec3 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:22.717249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 269/539] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:00:22 2022] POST /identity/v3/services => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:22.718588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5253-c5e1-4392-8953-d8dc943381bc None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.718879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5253-c5e1-4392-8953-d8dc943381bc None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.719199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5253-c5e1-4392-8953-d8dc943381bc None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.724834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 270/540] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:22.724834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5253-c5e1-4392-8953-d8dc943381bc None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:22.830765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d72609-00b4-4062-908e-46f0e4d891fa None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:22.833981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:22.873644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5253-c5e1-4392-8953-d8dc943381bc None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:22.882769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd46b5c-a34f-4e8b-bdfc-feec0f2f5dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:22.883454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd46b5c-a34f-4e8b-bdfc-feec0f2f5dab None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.883735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd46b5c-a34f-4e8b-bdfc-feec0f2f5dab None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.884020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd46b5c-a34f-4e8b-bdfc-feec0f2f5dab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.898229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd46b5c-a34f-4e8b-bdfc-feec0f2f5dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:22.899174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7cd46b5c-a34f-4e8b-bdfc-feec0f2f5dab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:22.904263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:22.905214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:22.906046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 271/541] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:00:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:22.907024 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a881ba-d150-469b-8d92-39b2734df6cc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.907144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a881ba-d150-469b-8d92-39b2734df6cc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.907354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a881ba-d150-469b-8d92-39b2734df6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.917874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9c14f-309e-424a-831b-acba689badad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:22.918553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c14f-309e-424a-831b-acba689badad None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.918842 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c14f-309e-424a-831b-acba689badad None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.919188 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9c14f-309e-424a-831b-acba689badad None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.920473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c14f-309e-424a-831b-acba689badad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:22.920473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c14f-309e-424a-831b-acba689badad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:22.921494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9c14f-309e-424a-831b-acba689badad None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:22.927491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-34a881ba-d150-469b-8d92-39b2734df6cc None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:22.928747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-34a881ba-d150-469b-8d92-39b2734df6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:22.935437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 271/542] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:22.936674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704641-d88d-4034-a0cc-f64ebeac0718 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:22.936955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704641-d88d-4034-a0cc-f64ebeac0718 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:22.937227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e704641-d88d-4034-a0cc-f64ebeac0718 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:22.943199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 272/543] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:22.943804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704641-d88d-4034-a0cc-f64ebeac0718 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.007200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 272/544] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:23.008046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a881ba-d150-469b-8d92-39b2734df6cc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.071592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e704641-d88d-4034-a0cc-f64ebeac0718 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.124112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:23.127620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96363c38-4c8f-42c4-917f-ff4f234cf4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:23.128752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96363c38-4c8f-42c4-917f-ff4f234cf4aa None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.129341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96363c38-4c8f-42c4-917f-ff4f234cf4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.129860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96363c38-4c8f-42c4-917f-ff4f234cf4aa None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.130734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96363c38-4c8f-42c4-917f-ff4f234cf4aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:23.131540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96363c38-4c8f-42c4-917f-ff4f234cf4aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:23.133802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96363c38-4c8f-42c4-917f-ff4f234cf4aa None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:23.136114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a881ba-d150-469b-8d92-39b2734df6cc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.152433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 273/545] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:23.152605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca32eee-978c-4e84-96e6-62c967d4bf60 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.153006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca32eee-978c-4e84-96e6-62c967d4bf60 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.153006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca32eee-978c-4e84-96e6-62c967d4bf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.172101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-2ca32eee-978c-4e84-96e6-62c967d4bf60 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:23.173406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2ca32eee-978c-4e84-96e6-62c967d4bf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:23.183785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:23.184423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.184705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.184978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.193350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:23.194803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:23.208012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:23.208978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:00:23.209897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:23.214925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 273/546] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:00:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:23.216011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.231376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:23.274370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 274/547] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 24 13:00:23.274370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca32eee-978c-4e84-96e6-62c967d4bf60 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.352733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd540d8-b5df-4d43-85be-1c2f7391c610 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.393305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:23.393962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.394253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.394535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.399801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca32eee-978c-4e84-96e6-62c967d4bf60 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.407303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:23.408885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:23.425152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:23.426162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:23.427079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:23.427079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:23.427079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:23.427761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 274/548] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:23.431082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.431935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:23.440303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:23.441036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.441318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.441595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.442366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:23.442741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:23.444356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:23.468601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 275/549] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 24 13:00:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:23.469456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.509381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:23.562438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851b08f-850e-48c7-9af4-224db750c6a2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.592131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdc027-658d-4287-a92b-b155bd5a4b4e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.606991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b234be2-172d-4d61-b7c1-18c3daf761e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:23.607945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234be2-172d-4d61-b7c1-18c3daf761e4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.608232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234be2-172d-4d61-b7c1-18c3daf761e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.608568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234be2-172d-4d61-b7c1-18c3daf761e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.619902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b234be2-172d-4d61-b7c1-18c3daf761e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5b234be2-172d-4d61-b7c1-18c3daf761e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:23.625950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:23.627525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 275/550] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:23.628872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b234be2-172d-4d61-b7c1-18c3daf761e4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.629625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:23.629990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.630405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.630570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.631399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:23.631725 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:23.633389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:23.647437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 276/551] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:23.648888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.706698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:23.747213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:23.750267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b234be2-172d-4d61-b7c1-18c3daf761e4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.758497 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09937b-3b27-4b58-b758-ddccc4342840 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.785343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:23.785968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.786290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.786527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.794521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61220355-67e9-45c9-8c07-9b85cfc4433f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:23.795158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220355-67e9-45c9-8c07-9b85cfc4433f None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:23.795451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220355-67e9-45c9-8c07-9b85cfc4433f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:23.795726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61220355-67e9-45c9-8c07-9b85cfc4433f None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:23.796376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61220355-67e9-45c9-8c07-9b85cfc4433f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:23.797500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:23.797754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61220355-67e9-45c9-8c07-9b85cfc4433f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:23.798943 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:23.814169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:23.815417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:23.816303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:23.816303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:23.816303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:23.816303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 276/552] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:23.816303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.816986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 277/553] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:00:23 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:23.817851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61220355-67e9-45c9-8c07-9b85cfc4433f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:23.950621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517bdc2-1d2d-42e7-b63b-a18079cf3d7a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:23.971664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:23.971994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61220355-67e9-45c9-8c07-9b85cfc4433f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.001797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.002583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.002875 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.003504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.004451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:24.004760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.006893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.011290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.011960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.013268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.013268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.016580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 277/554] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:24.017285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.025802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.027844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:24.043093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:24.044113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:24.045049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 278/555] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 24 13:00:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:24.045637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.059996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:24.159757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4c79f-440c-47c5-8309-a9d5aab4b49e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.178478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70330de0-9061-40be-8b98-ae6483111ab3 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.209028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.210462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.211158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.211709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.212555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:24.214082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.216129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.229008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.229843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 278/556] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:24.230427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.230557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.231039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.231562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.245219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.247825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:24.269607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:24.270777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:24.271964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:24.271964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:24.271964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:24.273351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 279/557] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:24.274263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.280297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:24.337655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:24.350691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a34f5d5-27c6-43f2-9d54-245184bc21ec None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.382259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02aaef-e5e8-4b02-ba92-960f00cdff2f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.389049 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.389786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.390194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.390560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.391783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:24.391783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.393202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.415744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 279/558] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:24.415744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.426089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.426974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.427475 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.427929 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.435829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.437558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:24.450979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:24.451926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:24.452799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:24.452799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:24.452799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:24.452799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:24.452799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:24.452799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:24.452799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:24.452799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:24.455225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 280/559] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:24.456178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.515826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:24.522337 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda36504-9cfe-42f1-8e98-8534b8c2f709 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.560351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.561214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.561674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.562098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.563093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:24.563904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.565725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.570424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d48c5-3bcf-4bd5-a4cd-a8391704c975 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.580429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 280/560] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:24.581485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.607510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:24.620477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b951b-1dc2-4a0c-9da1-9057c88d5c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.621526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b951b-1dc2-4a0c-9da1-9057c88d5c43 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.621911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b951b-1dc2-4a0c-9da1-9057c88d5c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.622290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b951b-1dc2-4a0c-9da1-9057c88d5c43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.695715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b951b-1dc2-4a0c-9da1-9057c88d5c43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.697934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b951b-1dc2-4a0c-9da1-9057c88d5c43 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.712472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838c77f-3942-4645-a876-5e031550fef0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.756216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.757214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.757648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.758131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.759015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:24.759821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.761444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.775461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 281/561] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:24.776770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.794192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:24.813241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 281/562] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:00:24.813241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b951b-1dc2-4a0c-9da1-9057c88d5c43 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.874489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:24.898817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33be7dc-f544-4c4f-88d6-f78e8c26fa95 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.918628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b951b-1dc2-4a0c-9da1-9057c88d5c43 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:24.935772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.936683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.937178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.937630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.945634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.947863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.954466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44471c54-93fb-436f-a718-0d028aa1cb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:24.955214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44471c54-93fb-436f-a718-0d028aa1cb2d None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.955453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44471c54-93fb-436f-a718-0d028aa1cb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.955733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44471c54-93fb-436f-a718-0d028aa1cb2d None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/5a63131a456548c8a3a3e85dbeba5a48/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.957669 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44471c54-93fb-436f-a718-0d028aa1cb2d None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=5a63131a456548c8a3a3e85dbeba5a48, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:24.959140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44471c54-93fb-436f-a718-0d028aa1cb2d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:24.961848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:24.962870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:00:24.964292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:24.967340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 282/563] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:24.968235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:24.975800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 282/564] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:24 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/5a63131a456548c8a3a3e85dbeba5a48/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:24.983012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d817-a8dc-4618-ac03-eecf65720a25 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:24.983695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d817-a8dc-4618-ac03-eecf65720a25 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:24.984644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d817-a8dc-4618-ac03-eecf65720a25 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:24.986970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 283/565] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:24.988093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791d817-a8dc-4618-ac03-eecf65720a25 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:25.078249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:25.110786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3d589-e473-4668-a53b-2edbe0e9f107 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.119665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:25.152894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791d817-a8dc-4618-ac03-eecf65720a25 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.158147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c405fd-7bec-4ec9-80af-f658a3648b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:25.158803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c405fd-7bec-4ec9-80af-f658a3648b8c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:25.159116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c405fd-7bec-4ec9-80af-f658a3648b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:25.159403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c405fd-7bec-4ec9-80af-f658a3648b8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:25.174345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c405fd-7bec-4ec9-80af-f658a3648b8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:25.176064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c405fd-7bec-4ec9-80af-f658a3648b8c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-26c405fd-7bec-4ec9-80af-f658a3648b8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:25.192575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:25.193561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:25.194466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:25.194466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:25.194466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:25.195183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 283/566] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:25.196511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bf0da-b08a-4bd2-b428-3d27c193e00e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:25.196861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bf0da-b08a-4bd2-b428-3d27c193e00e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:25.197756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bf0da-b08a-4bd2-b428-3d27c193e00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:25.202809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:25.203703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:25.204192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:25.204689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:25.205611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:25.206292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:25.207995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:25.217658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4f9bf0da-b08a-4bd2-b428-3d27c193e00e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:25.218856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4f9bf0da-b08a-4bd2-b428-3d27c193e00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:25.227818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 284/567] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Oct 24 13:00:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:25.228983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:25.304350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 284/568] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:25.311254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bf0da-b08a-4bd2-b428-3d27c193e00e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:25.345919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67163c1-1bc1-45f7-8a42-af8ba23fa9d0 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.401583 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:25.402586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:25.403071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:25.403579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:25.404408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:25.404895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:25.406814 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:25.419828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 285/569] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:25.420730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:25.424696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:25.425991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:25.504996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bf0da-b08a-4bd2-b428-3d27c193e00e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.566443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:25.575259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:25.575794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:25.576270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:25.591494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8cf30-bfc8-4f2a-b5b2-ddf6455c9cd6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.593830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:25.595542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:25.623227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:25.624205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:25.625117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:25.625117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:25.625117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:25.625117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 285/570] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:25.629642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:25.630176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:25.630820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:25.631396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:25.631671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:25.642191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:25.643700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:25.665273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:25.666290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:25.667199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 286/571] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:25.679634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:25.751549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:25.776907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318c2c9-275d-4a58-bb89-69dbddb81726 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.787165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:25.800748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e31da5-4075-4dc2-8119-846a9dfc6109 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.814606 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:25.815600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:25.816087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:25.816566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:25.817516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:25.818112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:25.819728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:25.836052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 286/572] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:25.837049 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:25.840805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:25.841429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:25.841735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:25.841990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:25.842682 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:25.842994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:25.844450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:25.862160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 287/573] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:25.862160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:25.968340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d967ee1-c59e-4545-b6b5-d047a2b2db1d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.969074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714b1b7-3a17-4f57-93f2-89c964e76594 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:25.993624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:26.014331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.015659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.016650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.017133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.017916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:26.018924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.019610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.019897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.020181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.020286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:26.029635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:26.031176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:26.033880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:26.034974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:26.036043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:26.038945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 287/574] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Oct 24 13:00:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:26.040146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:26.048021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:26.049254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:26.050420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:26.050420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:26.050420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:26.050420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:26.050420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:26.050420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:26.050420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:26.050420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:26.055419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 288/575] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:26.055419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:26.091630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:26.163727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82614218-2d75-423c-aba6-42ade36c987a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:26.173562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edab887-86a5-47e9-9ec6-8cf84f7a618f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:26.201752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.202402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.202622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.202897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.203661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=00530732ebe84dababafb9f399452940, user.id=2dff78b5eadc42608afbf2b14dd07e7a) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:26.204093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:26.205367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:26.215109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 288/576] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:26 2022] GET /identity/v3/role_assignments?user.id=2dff78b5eadc42608afbf2b14dd07e7a&scope.project.id=00530732ebe84dababafb9f399452940&role.id=11b74890c7484c6088dd78c106de8be4 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:26.215109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:26.224952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b08662-9554-4c5a-96d9-627608de53af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.225498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b08662-9554-4c5a-96d9-627608de53af None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.225779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b08662-9554-4c5a-96d9-627608de53af None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.226054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b08662-9554-4c5a-96d9-627608de53af None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.226692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b08662-9554-4c5a-96d9-627608de53af None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:26.226996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b08662-9554-4c5a-96d9-627608de53af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:26.228602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b08662-9554-4c5a-96d9-627608de53af None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:26.239113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 289/577] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 24 13:00:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:26.240983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9346d-d6e6-47f0-8606-def57814a0c5 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.240983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9346d-d6e6-47f0-8606-def57814a0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.241137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9346d-d6e6-47f0-8606-def57814a0c5 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.247473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 290/578] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:26.248550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9346d-d6e6-47f0-8606-def57814a0c5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:26.314766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:26.348494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:26.357042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153327e-a259-495e-bb84-f50148ec2319 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:26.400611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9346d-d6e6-47f0-8606-def57814a0c5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:26.405752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76986ddb-40a4-4f12-8a68-9a69e78a9b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.408356 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76986ddb-40a4-4f12-8a68-9a69e78a9b30 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.408655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76986ddb-40a4-4f12-8a68-9a69e78a9b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.408917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76986ddb-40a4-4f12-8a68-9a69e78a9b30 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.409637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76986ddb-40a4-4f12-8a68-9a69e78a9b30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:26.412698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76986ddb-40a4-4f12-8a68-9a69e78a9b30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:26.412698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76986ddb-40a4-4f12-8a68-9a69e78a9b30 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:26.432589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 289/579] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:26.433878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43f90-0a65-4d7b-9079-0231742a821e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.433878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43f90-0a65-4d7b-9079-0231742a821e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.434176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd43f90-0a65-4d7b-9079-0231742a821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.472227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cfd43f90-0a65-4d7b-9079-0231742a821e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:26.473502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cfd43f90-0a65-4d7b-9079-0231742a821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:26.473850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.474475 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.474760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.475123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.475808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:26.476118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:26.477690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:26.486257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 291/580] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 24 13:00:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:26.486257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:26.577772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 290/581] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:26.578648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd43f90-0a65-4d7b-9079-0231742a821e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:26.604994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bd71a-0f10-4967-902c-81d671c43b54 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:26.612839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:26.651156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866fe2-4fa8-45cd-bc13-6e08c533c07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.652262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866fe2-4fa8-45cd-bc13-6e08c533c07e None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.652749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866fe2-4fa8-45cd-bc13-6e08c533c07e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.653321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53866fe2-4fa8-45cd-bc13-6e08c533c07e None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/b2396a5c6002491f926326ee2d154f92/roles/6598739f382742d7b956c17b01862204` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.689459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53866fe2-4fa8-45cd-bc13-6e08c533c07e None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=b2396a5c6002491f926326ee2d154f92, role_id=6598739f382742d7b956c17b01862204)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:26.691488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53866fe2-4fa8-45cd-bc13-6e08c533c07e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:26.723640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:26.727618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 292/582] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:26 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/b2396a5c6002491f926326ee2d154f92/roles/6598739f382742d7b956c17b01862204 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:26.741740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd43f90-0a65-4d7b-9079-0231742a821e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:26.789584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.790225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.794178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.794482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.796342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] RBAC: Adding query filter params (interface=public, service_id=b859df0562954d6abfe46239bc3f63b2, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:26.798208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:26.800450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:26.821835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 291/583] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 24 13:00:26 2022] GET /identity/v3/endpoints?service_id=b859df0562954d6abfe46239bc3f63b2&interface=public®ion_id=RegionOne => generated 194 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:26.826275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866fe2-4fa8-45cd-bc13-6e08c533c07e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:26.950633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53866fe2-4fa8-45cd-bc13-6e08c533c07e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:26.996921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4352-4389-4094-9d99-4d04e5a3e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:26.997484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4352-4389-4094-9d99-4d04e5a3e7ce None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:26.997776 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4352-4389-4094-9d99-4d04e5a3e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:26.998313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4352-4389-4094-9d99-4d04e5a3e7ce None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:26.998945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b4352-4389-4094-9d99-4d04e5a3e7ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:27.000430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b4352-4389-4094-9d99-4d04e5a3e7ce None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:27.006941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 293/584] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:00:26 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:27.012840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:27.195533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a881c1-f49c-4197-aecf-57f18117fa65 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:27.238928 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b971a-dfd2-4d31-9ccc-ccfaf1ddbd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:27.239594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b971a-dfd2-4d31-9ccc-ccfaf1ddbd47 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:27.239877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b971a-dfd2-4d31-9ccc-ccfaf1ddbd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:27.240155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b971a-dfd2-4d31-9ccc-ccfaf1ddbd47 None admin] PATH_INFO: `/v3/services/network` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:27.240742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b971a-dfd2-4d31-9ccc-ccfaf1ddbd47 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:27.242111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b971a-dfd2-4d31-9ccc-ccfaf1ddbd47 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4e8b971a-dfd2-4d31-9ccc-ccfaf1ddbd47 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:27.251514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:27.252592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:27.253499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 292/585] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:27 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:27.262007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4352-4389-4094-9d99-4d04e5a3e7ce None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:27.288294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:27.377617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b4352-4389-4094-9d99-4d04e5a3e7ce None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:27.404150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56a6de-36bd-4d29-add6-bd8d23072c06 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:27.404150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56a6de-36bd-4d29-add6-bd8d23072c06 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:27.404150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56a6de-36bd-4d29-add6-bd8d23072c06 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:27.405553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 293/586] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:27.410811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6604968-293c-4adc-b298-e8147efa57d8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:27.411375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6604968-293c-4adc-b298-e8147efa57d8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:27.411945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6604968-293c-4adc-b298-e8147efa57d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:27.418074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:27.418695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:27.418981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:27.419290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:27.419918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] RBAC: Adding query filter params (name=network) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:27.420225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:27.421600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:27.427809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 294/587] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:27.433581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:27.440491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c6604968-293c-4adc-b298-e8147efa57d8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:27.441785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c6604968-293c-4adc-b298-e8147efa57d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:27.459476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:27.556845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:27.573672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7d105-fc50-427f-bb3c-d514ec7fe9f3 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:27.614767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 294/588] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:27.618878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:27.619539 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:27.619827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:27.620106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:27.620756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] RBAC: Adding query filter params (type=network) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:27.621063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:27.622462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:27.631506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 295/589] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:27.643917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6604968-293c-4adc-b298-e8147efa57d8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:27.786826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6604968-293c-4adc-b298-e8147efa57d8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:27.821732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:27.823027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:27.823194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:27.823418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:27.824128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] RBAC: Adding query filter params (interface=public, service_id=661aa35b08b649e8a633037d027921b0, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:27.824441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:27.825823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:27.831295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 295/590] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 24 13:00:27 2022] GET /identity/v3/endpoints?service_id=661aa35b08b649e8a633037d027921b0&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:27.836976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:27.989905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb239d-1b1f-416b-aa93-a5893f400449 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:27.994745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:28.015410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:28.042176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5dae9-cf02-4e8a-b930-4df093b807ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:28.043238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5dae9-cf02-4e8a-b930-4df093b807ae None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:28.045687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5dae9-cf02-4e8a-b930-4df093b807ae None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:28.046240 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5dae9-cf02-4e8a-b930-4df093b807ae None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:28.047126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5dae9-cf02-4e8a-b930-4df093b807ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:28.050223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5dae9-cf02-4e8a-b930-4df093b807ae None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:28.065126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 296/591] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:00:27 2022] GET /identity/v3/services => generated 772 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:28.149612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7e80f-76ac-408c-aca5-e303c5cd4df7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:28.190437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531ef61-ee89-462f-9165-8fa7be8aeb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:28.191239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531ef61-ee89-462f-9165-8fa7be8aeb38 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:28.191525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531ef61-ee89-462f-9165-8fa7be8aeb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:28.191805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531ef61-ee89-462f-9165-8fa7be8aeb38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:28.213978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8531ef61-ee89-462f-9165-8fa7be8aeb38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8531ef61-ee89-462f-9165-8fa7be8aeb38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:28.220240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:28.221202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:28.222030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 296/592] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:28.227545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5dae9-cf02-4e8a-b930-4df093b807ae None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:28.260093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:28.374702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5dae9-cf02-4e8a-b930-4df093b807ae None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:28.413206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:28.413839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:28.414126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:28.414402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:28.415096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:28.415537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:28.417051 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:28.426349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 297/593] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:28.432310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531ef61-ee89-462f-9165-8fa7be8aeb38 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:28.519985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:28.564793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8531ef61-ee89-462f-9165-8fa7be8aeb38 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:28.606241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:28.606880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:28.607193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:28.607473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:28.622889 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:28.624396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:28.642428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:28.643557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:28.646108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:28.646108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:28.646108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:28.651082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 297/594] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:28.651314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:28.786319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:28.816619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915c28a-0aea-48f4-9992-f1f2270aed73 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:28.871156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c11e9b-34e5-4e61-bd30-6a9360eaedda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:28.872588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c11e9b-34e5-4e61-bd30-6a9360eaedda None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:28.873083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c11e9b-34e5-4e61-bd30-6a9360eaedda None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:28.873564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c11e9b-34e5-4e61-bd30-6a9360eaedda None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:28.874428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c11e9b-34e5-4e61-bd30-6a9360eaedda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:28.874941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c11e9b-34e5-4e61-bd30-6a9360eaedda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:28.876537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c11e9b-34e5-4e61-bd30-6a9360eaedda None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:28.896339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 298/595] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:28.902515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:29.064598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e2fb-5eff-4c10-bf20-f517e975c0d8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:29.095438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:29.117059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:29.117699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.117987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.118269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.126701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:29.128191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:29.144091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:29.145120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 24 13:00:29.146154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:29.148761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 298/596] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:00:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:29.155078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658e021-2000-4542-8dac-8afa10e17715 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.155078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658e021-2000-4542-8dac-8afa10e17715 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.155078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658e021-2000-4542-8dac-8afa10e17715 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.155078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 299/597] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:29.156535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:29.158648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e84499-632b-4f05-bce3-1a1443f88ca0 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.158939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e84499-632b-4f05-bce3-1a1443f88ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.159267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e84499-632b-4f05-bce3-1a1443f88ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.177537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d5e84499-632b-4f05-bce3-1a1443f88ca0 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:29.180297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d5e84499-632b-4f05-bce3-1a1443f88ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:29.263790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 300/598] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:29.343314 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a59710-2ff4-48ad-b3b1-071187141942 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:29.400687 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:29.401667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.402174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.402661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.403657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:29.404198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:29.405974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:29.441360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 299/599] 10.222.0.42 () {56 vars in 1135 bytes} [Mon Oct 24 13:00:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:29.448279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e84499-632b-4f05-bce3-1a1443f88ca0 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:29.544161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:29.593795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e84499-632b-4f05-bce3-1a1443f88ca0 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:29.612561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:29.634255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f56b5-1bbc-4b07-be86-6331119ba322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:29.635793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f56b5-1bbc-4b07-be86-6331119ba322 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.636238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f56b5-1bbc-4b07-be86-6331119ba322 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.636655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f56b5-1bbc-4b07-be86-6331119ba322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.669540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:29.735420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f56b5-1bbc-4b07-be86-6331119ba322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:29.738237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4f56b5-1bbc-4b07-be86-6331119ba322 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:29.747486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cac9b3-e142-4532-b735-dc9eaafc4a72 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:29.792532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9e503-bf73-4c59-af0b-99fb502fb244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:29.793415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9e503-bf73-4c59-af0b-99fb502fb244 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.794014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9e503-bf73-4c59-af0b-99fb502fb244 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.794441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9e503-bf73-4c59-af0b-99fb502fb244 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.795319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9e503-bf73-4c59-af0b-99fb502fb244 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:29.796933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9e503-bf73-4c59-af0b-99fb502fb244 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f4d9e503-bf73-4c59-af0b-99fb502fb244 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:29.809674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:29.810884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:29.812015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:29.819916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 300/600] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 24 13:00:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:29.821191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:29.823834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4f11-68ae-4ae4-b6d8-12d5a648bd96 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.824374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4f11-68ae-4ae4-b6d8-12d5a648bd96 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.824812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4f11-68ae-4ae4-b6d8-12d5a648bd96 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.826777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 301/601] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:29.827707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb4f11-68ae-4ae4-b6d8-12d5a648bd96 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:29.868297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 301/602] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:00:29.869586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0ae74-9e5c-405c-9d0f-f14b0e74bbbd None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.869871 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0ae74-9e5c-405c-9d0f-f14b0e74bbbd None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.870145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0ae74-9e5c-405c-9d0f-f14b0e74bbbd None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.871582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 302/603] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:29.876517 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b923b3-9252-4a2b-bd38-232e0a626c4f None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.876805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b923b3-9252-4a2b-bd38-232e0a626c4f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.877078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b923b3-9252-4a2b-bd38-232e0a626c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.895425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c1b923b3-9252-4a2b-bd38-232e0a626c4f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:29.896609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c1b923b3-9252-4a2b-bd38-232e0a626c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:29.948595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb4f11-68ae-4ae4-b6d8-12d5a648bd96 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:29.950916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:29.966904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 303/604] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:29.967772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b923b3-9252-4a2b-bd38-232e0a626c4f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:29.990377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a97ee-fdc3-4c86-b853-6eff079a5ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:29.990977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a97ee-fdc3-4c86-b853-6eff079a5ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:29.991296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a97ee-fdc3-4c86-b853-6eff079a5ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:29.991568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a97ee-fdc3-4c86-b853-6eff079a5ae2 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:29.992192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a97ee-fdc3-4c86-b853-6eff079a5ae2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:29.992525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a97ee-fdc3-4c86-b853-6eff079a5ae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:29.994394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a97ee-fdc3-4c86-b853-6eff079a5ae2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:30.003426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 302/605] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:30.004645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a224e3a-e3da-427d-9f4b-d90c5ccb40e6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.004937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a224e3a-e3da-427d-9f4b-d90c5ccb40e6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.005217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a224e3a-e3da-427d-9f4b-d90c5ccb40e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.024624 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1a224e3a-e3da-427d-9f4b-d90c5ccb40e6 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:30.026034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1a224e3a-e3da-427d-9f4b-d90c5ccb40e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:30.100977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:30.109453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b923b3-9252-4a2b-bd38-232e0a626c4f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:30.123080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 303/606] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:30.123080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a224e3a-e3da-427d-9f4b-d90c5ccb40e6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:30.169469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:30.170104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.170404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.170651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.191241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:30.192685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:30.219012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:30.220012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:30.220918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:30.220918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:30.220918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:30.221087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 304/607] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:30.228731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:30.281007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:30.288674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a224e3a-e3da-427d-9f4b-d90c5ccb40e6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:30.329965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:30.331385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.331987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.332563 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.333395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:30.333895 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:30.335702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:30.341773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 304/608] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:30.342835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:30.402820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6529d6-49fd-42f8-91b1-1e97a23c220d None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:30.459974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:30.460595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.460882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.461156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.461804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:30.462107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:30.464355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:30.482643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 305/609] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:30.484086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:30.498745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:30.499882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cde8da-9512-44ea-b550-a6b95a218d1b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:30.547381 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c1b18-79b1-46db-b505-b984bd25ea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:30.548035 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c1b18-79b1-46db-b505-b984bd25ea28 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.548316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c1b18-79b1-46db-b505-b984bd25ea28 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.548593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c1b18-79b1-46db-b505-b984bd25ea28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.563674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c1b18-79b1-46db-b505-b984bd25ea28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ef7c1b18-79b1-46db-b505-b984bd25ea28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:30.568973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:30.570007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:30.570855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 305/610] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:30.571622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c1b18-79b1-46db-b505-b984bd25ea28 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:30.595918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:30.628576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9578f75b-71f1-4951-919e-9839844c130b None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:30.686362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:30.688309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.688839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.689396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.690054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c1b18-79b1-46db-b505-b984bd25ea28 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:30.690449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:30.692508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:30.707446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:30.708516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 24 13:00:30.709385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:30.711989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 306/611] 10.222.0.42 () {56 vars in 1129 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:30.714192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:30.735235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:30.736213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.736637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.737052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.746038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:30.747932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:30.764012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:30.764962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:30.766241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 306/612] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:30.771455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:30.815797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:30.851304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c26eb-e125-42db-a8d0-579dbe445e71 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:30.869862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:30.887893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:30.888644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.888935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.889219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.889885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:30.890190 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:30.891601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:30.892469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e772940-0421-47cb-8905-b8cc2fca7f6d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:30.901212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 307/613] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:30.901654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:30.933562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa0734-986b-4107-a5d3-48293a4ce0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:30.934174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0734-986b-4107-a5d3-48293a4ce0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.934453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0734-986b-4107-a5d3-48293a4ce0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.934724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa0734-986b-4107-a5d3-48293a4ce0b6 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.935383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa0734-986b-4107-a5d3-48293a4ce0b6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:30.935686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa0734-986b-4107-a5d3-48293a4ce0b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:30.937048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa0734-986b-4107-a5d3-48293a4ce0b6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:30.945854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 307/614] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:30.945854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eaec4-10b3-4919-9f54-5c02e1056784 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:30.945854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eaec4-10b3-4919-9f54-5c02e1056784 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:30.945854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853eaec4-10b3-4919-9f54-5c02e1056784 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:30.945854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 308/615] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:30.945854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eaec4-10b3-4919-9f54-5c02e1056784 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.013638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de24faa-10f1-4ed3-9aaa-b24e2938cc0d None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.043435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:31.049445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.050081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.050356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.050630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.051343 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:31.051652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.053085 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.059379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853eaec4-10b3-4919-9f54-5c02e1056784 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.067380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 308/616] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:31.068231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.108955 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3879637-8fc3-4cc0-a9f2-43847fe694e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.109591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879637-8fc3-4cc0-a9f2-43847fe694e1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.109872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879637-8fc3-4cc0-a9f2-43847fe694e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.110146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3879637-8fc3-4cc0-a9f2-43847fe694e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.120832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3879637-8fc3-4cc0-a9f2-43847fe694e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.122346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3879637-8fc3-4cc0-a9f2-43847fe694e1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.125005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a3879637-8fc3-4cc0-a9f2-43847fe694e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:31.140996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:31.142087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:31.142987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:31.142987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:31.142987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:31.142987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 309/617] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:31.143714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7d472-c259-44bd-a80e-42e3ac430bdb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.143999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7d472-c259-44bd-a80e-42e3ac430bdb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.144273 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7d472-c259-44bd-a80e-42e3ac430bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.162797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-10a7d472-c259-44bd-a80e-42e3ac430bdb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:31.164015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-10a7d472-c259-44bd-a80e-42e3ac430bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:31.179286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738ab7e-150a-42b1-a198-63e28575cca1 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.214643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.215283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.215565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.215846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.216494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:31.216785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.218159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.227326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 309/618] 10.222.0.42 () {56 vars in 1124 bytes} [Mon Oct 24 13:00:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:31.227326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.240057 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 310/619] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:31.240929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7d472-c259-44bd-a80e-42e3ac430bdb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.292589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:31.343509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42889784-65cc-4ae4-a29f-f46372c7cdb5 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.362949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7d472-c259-44bd-a80e-42e3ac430bdb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.379772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.380463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.380816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.381733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.382199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=00530732ebe84dababafb9f399452940, user.id=5a63131a456548c8a3a3e85dbeba5a48) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:31.382576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.384101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.399097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 310/620] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:31 2022] GET /identity/v3/role_assignments?user.id=5a63131a456548c8a3a3e85dbeba5a48&scope.project.id=00530732ebe84dababafb9f399452940&role.id=11b74890c7484c6088dd78c106de8be4 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:31.399097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.414779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:31.432418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.433138 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.433426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.433706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.434370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:31.434683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.436294 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.468374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 311/621] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:31.476211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.593463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87167f5c-b952-49ca-afb0-c75cae409968 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.598913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe9ed1-90ec-4cbf-88dc-8882330d2b94 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.615977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:31.631323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae1aee-2881-4fe6-b3a0-cd8c97eca637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.631942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1aee-2881-4fe6-b3a0-cd8c97eca637 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.632245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1aee-2881-4fe6-b3a0-cd8c97eca637 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.632494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae1aee-2881-4fe6-b3a0-cd8c97eca637 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.633153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae1aee-2881-4fe6-b3a0-cd8c97eca637 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.635681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae1aee-2881-4fe6-b3a0-cd8c97eca637 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.641080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.641722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.642004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.642278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.652586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.654165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.660889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 311/622] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:00:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:31.661641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae1aee-2881-4fe6-b3a0-cd8c97eca637 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:31.667252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:31.668453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:00:31.669407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:31.669930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 312/623] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 24 13:00:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:31.675775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.764776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:31.841048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b22e35-4be7-4975-a414-b7069d537099 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.842792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae1aee-2881-4fe6-b3a0-cd8c97eca637 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:31.879241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.880200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.880707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.881463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.882411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:31.882930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.885300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.905708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 313/624] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Oct 24 13:00:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:31.910978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:31.912224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:31.913288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.914560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:31.915074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] PATH_INFO: `/v3/services/network` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:31.916272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:31.919347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:31.932220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:31.933220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 24 13:00:31.934054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:31.936903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 312/625] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:31 2022] GET /identity/v3/services/network => generated 88 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:31.942079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:31.990616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:32.045719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5156aa0-598d-4443-beb2-8a41e95be50e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:32.048546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:32.076342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4199329c-0d72-4bf2-8278-18de9cce1168 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:32.088045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50798-9253-4d19-8182-55ec3eedd601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:32.089338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50798-9253-4d19-8182-55ec3eedd601 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.089776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50798-9253-4d19-8182-55ec3eedd601 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.090255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d50798-9253-4d19-8182-55ec3eedd601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:32.116224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.116624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.117000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.117911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] RBAC: Adding query filter params (name=network) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:32.118137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:32.120578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:32.125474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 313/626] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:32.132144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:32.178091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50798-9253-4d19-8182-55ec3eedd601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:32.179873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d50798-9253-4d19-8182-55ec3eedd601 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:32.252688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:32.257387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d430932-4eed-4174-980a-4c4772fbb383 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:32.316013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 314/627] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:00:32.323470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50798-9253-4d19-8182-55ec3eedd601 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:32.327122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:32.333830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:32.334464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.334742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.335013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.335665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] RBAC: Adding query filter params (type=network) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:32.336229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:32.338333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:32.349512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 314/628] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:32.355805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:32.496673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41cd3d-4da0-497a-91f5-a963be6c105c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:32.510383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d50798-9253-4d19-8182-55ec3eedd601 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:32.545998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:32.546921 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.547573 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.547999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.560273 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:32.562123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:32.578464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d18c4-7bdc-400e-a0d6-09f208acf81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:32.579101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d18c4-7bdc-400e-a0d6-09f208acf81b None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.579393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d18c4-7bdc-400e-a0d6-09f208acf81b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.579670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d18c4-7bdc-400e-a0d6-09f208acf81b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.580353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d18c4-7bdc-400e-a0d6-09f208acf81b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:32.580962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:32.582163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:32.583291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:32.583291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:32.583291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:32.583511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d18c4-7bdc-400e-a0d6-09f208acf81b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:32.585108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 315/629] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:32.591595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:32.611084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 315/630] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:00:32 2022] POST /identity/v3/endpoints => generated 337 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:00:32.673340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:32.724681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db84e06-b52f-4366-9e02-a2971c5ff9ec None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:32.769065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:32.769600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.769884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.770160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.770810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:32.771146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:32.772552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:32.786315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 316/631] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:32.787814 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f6e15-da8a-48b8-95a6-f64637fcef84 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.788267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f6e15-da8a-48b8-95a6-f64637fcef84 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.788698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f6e15-da8a-48b8-95a6-f64637fcef84 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.790265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 316/632] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:32.792399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:32.796629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5372a-0389-45f9-b509-f7a7e78347bf None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.797040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5372a-0389-45f9-b509-f7a7e78347bf None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.797416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5372a-0389-45f9-b509-f7a7e78347bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.817273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-dfd5372a-0389-45f9-b509-f7a7e78347bf None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:32.818642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-dfd5372a-0389-45f9-b509-f7a7e78347bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:32.840045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:32.924319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 317/633] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:32.943672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f5679-cc49-4b5f-94d6-cd431d3aff55 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:32.960039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:32.983913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:32.984635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:32.984985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:32.985331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:32.992787 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:32.995108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:33.006866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:33.007990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:33.009210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:33.009210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:33.009210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:33.009210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:33.009210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:33.009210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:33.009210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:33.009210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:33.011171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 317/634] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:33.017639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5372a-0389-45f9-b509-f7a7e78347bf None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:33.152366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5372a-0389-45f9-b509-f7a7e78347bf None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:33.192233 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:33.193162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:33.193624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:33.194092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:33.194966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:33.195479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:33.197752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:33.212081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 318/635] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:33.218201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:33.238616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:33.253310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:33.348062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f4ed-e203-4900-af5a-3e3a239ae4a2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:33.351753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e7ca7-a6dc-40ac-b2b6-e42168ac064b None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:33.391099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a6a09-0db5-4092-ae13-02071e7ae88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:33.391729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6a09-0db5-4092-ae13-02071e7ae88b None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:33.392011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6a09-0db5-4092-ae13-02071e7ae88b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:33.392290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a6a09-0db5-4092-ae13-02071e7ae88b None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:33.392815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa59e9-599e-4f50-91f4-bf2db33c83dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:33.392949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a6a09-0db5-4092-ae13-02071e7ae88b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:33.393745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa59e9-599e-4f50-91f4-bf2db33c83dc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:33.394227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa59e9-599e-4f50-91f4-bf2db33c83dc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:33.394343 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a6a09-0db5-4092-ae13-02071e7ae88b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:33.394813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa59e9-599e-4f50-91f4-bf2db33c83dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:33.395766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa59e9-599e-4f50-91f4-bf2db33c83dc None admin] RBAC: Adding query filter params (role.id=6598739f382742d7b956c17b01862204, scope.project.id=00530732ebe84dababafb9f399452940, user.id=b2396a5c6002491f926326ee2d154f92) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:33.396281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa59e9-599e-4f50-91f4-bf2db33c83dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:33.398104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa59e9-599e-4f50-91f4-bf2db33c83dc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:33.415556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 318/636] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:33 2022] GET /identity/v3/role_assignments?user.id=b2396a5c6002491f926326ee2d154f92&scope.project.id=00530732ebe84dababafb9f399452940&role.id=6598739f382742d7b956c17b01862204 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:33.416158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 319/637] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:00:33 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:00:34.061154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f50a-fec7-4e31-9a7e-416afd3aaadb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:34.061509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f50a-fec7-4e31-9a7e-416afd3aaadb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:34.061728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524f50a-fec7-4e31-9a7e-416afd3aaadb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:34.063365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 319/638] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:34.068621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8cb81-1c08-4d2e-b8a3-eb68bc6a4985 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:34.069067 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8cb81-1c08-4d2e-b8a3-eb68bc6a4985 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:34.069249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8cb81-1c08-4d2e-b8a3-eb68bc6a4985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:34.090445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9cc8cb81-1c08-4d2e-b8a3-eb68bc6a4985 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:34.091672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9cc8cb81-1c08-4d2e-b8a3-eb68bc6a4985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:34.159658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:34.177146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 320/639] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:34 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:34.380729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd3406-c1d5-412f-af59-fb417441e417 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:34.381970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd3406-c1d5-412f-af59-fb417441e417 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:34.382492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd3406-c1d5-412f-af59-fb417441e417 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:34.385009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 320/640] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:34.390672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680f993-cf83-4982-8934-f7d8ab0a0232 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:34.391229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680f993-cf83-4982-8934-f7d8ab0a0232 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:34.391712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1680f993-cf83-4982-8934-f7d8ab0a0232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:34.410136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1680f993-cf83-4982-8934-f7d8ab0a0232 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:34.411624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1680f993-cf83-4982-8934-f7d8ab0a0232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:34.448997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd3406-c1d5-412f-af59-fb417441e417 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:34.487443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:34.507106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 321/641] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:34 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:34.562904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd3406-c1d5-412f-af59-fb417441e417 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:34.598708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:34.599591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:34.600044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:34.600468 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:34.601233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:34.602773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:34.612766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:34.614006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:34.615025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:34.617393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 321/642] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:34.623648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680f993-cf83-4982-8934-f7d8ab0a0232 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:34.736879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1680f993-cf83-4982-8934-f7d8ab0a0232 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:34.764868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:34.774160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:34.774851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:34.775785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:34.776305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:34.776920 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:34.777586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:34.779371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:34.784542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 322/643] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:34.790022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:34.813872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:34.913135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:34.916916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d049536-bc51-486f-935c-08ddf9cbf0bd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:34.926036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e5573-5eea-4f10-bdad-84d1a7994f35 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:34.958471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:34.959111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:34.959628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:34.959915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:34.960635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:34.960962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:34.963093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:34.965052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:34.966017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:34.966516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:34.966993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:34.967836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:34.972667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 322/644] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:34.972819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:34.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:34.983801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:34.984676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 323/645] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:34.992396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:35.120077 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac207d36-0541-4040-a772-7525f717cc9c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:35.157878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cd32d-1a85-468a-b8cf-9ff9f1f483ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:35.158488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cd32d-1a85-468a-b8cf-9ff9f1f483ca None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:35.158770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cd32d-1a85-468a-b8cf-9ff9f1f483ca None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:35.159043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cd32d-1a85-468a-b8cf-9ff9f1f483ca None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:35.159701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cd32d-1a85-468a-b8cf-9ff9f1f483ca None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:35.160007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cd32d-1a85-468a-b8cf-9ff9f1f483ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:35.161363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cd32d-1a85-468a-b8cf-9ff9f1f483ca None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:35.166883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 323/646] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:35.172812 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:35.232464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:35.280486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f14538-6e00-451b-853d-2ef473acfdde None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:35.316537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a683469-5130-4485-9a38-89fa88415556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:35.317154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a683469-5130-4485-9a38-89fa88415556 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:35.317438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a683469-5130-4485-9a38-89fa88415556 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:35.317722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a683469-5130-4485-9a38-89fa88415556 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:35.318493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a683469-5130-4485-9a38-89fa88415556 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:35.318802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a683469-5130-4485-9a38-89fa88415556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:35.320293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a683469-5130-4485-9a38-89fa88415556 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:35.326244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 324/647] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:35.669765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f598341-59ad-4964-a1d6-e2d46b4029c4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:35.670286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f598341-59ad-4964-a1d6-e2d46b4029c4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:35.670286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f598341-59ad-4964-a1d6-e2d46b4029c4 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:35.671920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 324/648] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:35.677101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8b291-3e01-4639-81f8-040a76936e14 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:35.677393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8b291-3e01-4639-81f8-040a76936e14 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:35.677666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8b291-3e01-4639-81f8-040a76936e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:35.699339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633481-cbb5-4bb8-aeaf-b1fa1343689b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:35.699643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633481-cbb5-4bb8-aeaf-b1fa1343689b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:35.699898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4633481-cbb5-4bb8-aeaf-b1fa1343689b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:35.701294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4cd8b291-3e01-4639-81f8-040a76936e14 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:35.701564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 325/649] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:35.702887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4cd8b291-3e01-4639-81f8-040a76936e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:35.715872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738572e2-49c1-4391-b1a0-2b48b951f890 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:35.716821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738572e2-49c1-4391-b1a0-2b48b951f890 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:35.717302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738572e2-49c1-4391-b1a0-2b48b951f890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:35.745995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-738572e2-49c1-4391-b1a0-2b48b951f890 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:35.747449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-738572e2-49c1-4391-b1a0-2b48b951f890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:35.778520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:35.792627 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 325/650] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:35.838662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 326/651] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:36.081325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8b291-3e01-4639-81f8-040a76936e14 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.108261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738572e2-49c1-4391-b1a0-2b48b951f890 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.135798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:36.195075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8b291-3e01-4639-81f8-040a76936e14 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.222939 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738572e2-49c1-4391-b1a0-2b48b951f890 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.232155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570ea64-3070-45f2-bd14-50a0450c3e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:36.232861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ea64-3070-45f2-bd14-50a0450c3e6f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:36.233144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ea64-3070-45f2-bd14-50a0450c3e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:36.233519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570ea64-3070-45f2-bd14-50a0450c3e6f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:36.244937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570ea64-3070-45f2-bd14-50a0450c3e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-f570ea64-3070-45f2-bd14-50a0450c3e6f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:36.250221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:36.251183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:00:36.252016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:36.256558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 326/652] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:36.258450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570ea64-3070-45f2-bd14-50a0450c3e6f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.262091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:36.262750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:36.263032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:36.263336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:36.264072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:36.265612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:36.276663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:36.277693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:36.279256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:36.283152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 327/653] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:36.284659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.314350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:36.394271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570ea64-3070-45f2-bd14-50a0450c3e6f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.400779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba1368-54d4-46cb-a6e8-e512e88ec516 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.406473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:36.439600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:36.440570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:36.440696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:36.441249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:36.441372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:36.441532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:36.441800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:36.442189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:36.442454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:36.442754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:36.443454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:36.443959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:36.444161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:36.445807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:36.453948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 327/654] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:36.460344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.460595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 328/655] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:36.466701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.564320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:36.577323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042700c8-a845-4b29-91c7-b8efb376aae4 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.581524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9208d-1596-47af-8fd6-ad541186d18f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.620899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:36.621508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:36.621789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:36.622062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:36.622692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:36.622995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:36.624556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:36.626161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:36.626903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:36.627228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:36.627506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:36.631215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 329/656] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:36.636193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.645167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:36.646560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:36.663235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:36.664223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:36.665127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:36.665127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:36.665127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:36.669863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 328/657] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:36.670804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.698226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:36.763560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df2d8e-5675-4ae9-a3ff-f85b1b27cac5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.787138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb37efb-6074-488f-ad67-ce42e421a885 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.805461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:36.806127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:36.806412 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:36.806740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:36.807596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] RBAC: Adding query filter params (interface=public, service_id=421f9d9774814b08b61f8b395632bf78, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:36.807830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:36.809203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:36.815090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 330/658] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/endpoints?service_id=421f9d9774814b08b61f8b395632bf78&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:36.820111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.838213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:36.843504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:36.844342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:36.844621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:36.844895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:36.845541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:36.845843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:36.847249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:36.860261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 329/659] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:36.865857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:36.952798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:36.956086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c4c99-7173-48ba-a57e-8fc2853a512b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:36.973134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb0dd8-5127-4930-8d9d-53d032ba93ac None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:37.000280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad81b9-6e87-4696-a87d-2f3ac6e1c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:37.001292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad81b9-6e87-4696-a87d-2f3ac6e1c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.001655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad81b9-6e87-4696-a87d-2f3ac6e1c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.002003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad81b9-6e87-4696-a87d-2f3ac6e1c1e8 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.002958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad81b9-6e87-4696-a87d-2f3ac6e1c1e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:37.004591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad81b9-6e87-4696-a87d-2f3ac6e1c1e8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:37.010697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c525df-0e5d-4192-bcde-122385f59ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:37.011700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c525df-0e5d-4192-bcde-122385f59ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.012166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c525df-0e5d-4192-bcde-122385f59ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.012619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c525df-0e5d-4192-bcde-122385f59ae8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.015125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 331/660] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/services => generated 1029 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:00:37.028144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c525df-0e5d-4192-bcde-122385f59ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:37.030117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c525df-0e5d-4192-bcde-122385f59ae8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-65c525df-0e5d-4192-bcde-122385f59ae8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:37.044266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:37.045276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:00:37.046212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:37.048910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 330/661] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 24 13:00:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:37.055200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad81b9-6e87-4696-a87d-2f3ac6e1c1e8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:37.149185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:37.177109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad81b9-6e87-4696-a87d-2f3ac6e1c1e8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:37.216478 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:37.217011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.217291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.217565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.218297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:37.218601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:37.220058 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:37.233865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21572c5c-d950-4c7f-b9df-ea44d098ea8b None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.234365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21572c5c-d950-4c7f-b9df-ea44d098ea8b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.234442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21572c5c-d950-4c7f-b9df-ea44d098ea8b None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.236056 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 331/662] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:37.239797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 332/663] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Oct 24 13:00:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:37.241478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ee7a-16d2-44da-a1a4-7589170c1dd8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.241765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ee7a-16d2-44da-a1a4-7589170c1dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.242040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260ee7a-16d2-44da-a1a4-7589170c1dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.245964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:37.263525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5260ee7a-16d2-44da-a1a4-7589170c1dd8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:37.264821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5260ee7a-16d2-44da-a1a4-7589170c1dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:37.342094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 332/664] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:37 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:37.367558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6b925-bdb8-44d2-b7db-826220339628 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:37.405254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:37.405871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.406157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.406436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.409259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:37.417707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:37.419244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:37.435706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:37.436693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:37.437664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:37.437664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:37.437664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:37.437664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 333/665] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:37.444136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260ee7a-16d2-44da-a1a4-7589170c1dd8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:37.529472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:37.561829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260ee7a-16d2-44da-a1a4-7589170c1dd8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:37.597789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b3624-225e-4a55-adab-8aa67a8ac414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:37.598410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3624-225e-4a55-adab-8aa67a8ac414 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.598697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3624-225e-4a55-adab-8aa67a8ac414 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.598975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b3624-225e-4a55-adab-8aa67a8ac414 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.599660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3624-225e-4a55-adab-8aa67a8ac414 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:37.599970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3624-225e-4a55-adab-8aa67a8ac414 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:37.601345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b3624-225e-4a55-adab-8aa67a8ac414 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:37.613408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 333/666] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:37.620150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:37.622063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9041e-3a1b-4d71-8b2b-bdb305f58fb5 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.622357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9041e-3a1b-4d71-8b2b-bdb305f58fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.622639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9041e-3a1b-4d71-8b2b-bdb305f58fb5 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.625264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 334/667] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:37.629261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f8d44-dc65-47b9-beda-6ffffd4e167b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.629549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f8d44-dc65-47b9-beda-6ffffd4e167b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.629831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f8d44-dc65-47b9-beda-6ffffd4e167b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.651687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6f0f8d44-dc65-47b9-beda-6ffffd4e167b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:37.653039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6f0f8d44-dc65-47b9-beda-6ffffd4e167b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:37.724575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 335/668] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:37 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:37.725401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f8d44-dc65-47b9-beda-6ffffd4e167b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:37.746704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189adffc-f60d-47e4-b2d2-a7d22ac4367b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:37.784629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:37.785255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.785494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.785723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.793077 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:37.794589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:37.806549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:37.807666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:37.813661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 334/669] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:37.814140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:37.815163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:37.868104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f8d44-dc65-47b9-beda-6ffffd4e167b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:37.873173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:37.906403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8745bb-f30b-48fa-867a-bc64a007c5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:37.907362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8745bb-f30b-48fa-867a-bc64a007c5ee None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.907861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8745bb-f30b-48fa-867a-bc64a007c5ee None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.908315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8745bb-f30b-48fa-867a-bc64a007c5ee None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.909137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8745bb-f30b-48fa-867a-bc64a007c5ee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:37.910674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8745bb-f30b-48fa-867a-bc64a007c5ee None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:37.931579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bec297-d18e-4489-833d-05bae198ff8e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:37.932120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 336/670] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:00:37 2022] POST /identity/v3/services => generated 241 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:37.978012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:37.978837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:37.979219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:37.979592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:37.980416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:37.980798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:37.982514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:37.998871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 335/671] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:38.005173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8745bb-f30b-48fa-867a-bc64a007c5ee None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:38.032010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:38.128028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8745bb-f30b-48fa-867a-bc64a007c5ee None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:38.155077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:38.156954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83e0af-2f3e-46af-8374-addc0a1c5d38 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:38.160811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:38.169143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e63888-f3b8-4e33-be0c-0454ad0c8d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:38.169511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e63888-f3b8-4e33-be0c-0454ad0c8d1e None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:38.169838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e63888-f3b8-4e33-be0c-0454ad0c8d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:38.170070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e63888-f3b8-4e33-be0c-0454ad0c8d1e None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/b2396a5c6002491f926326ee2d154f92/roles/7a6c27e9bb1d4ab0ae847254d4c8d5de` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:38.172314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e63888-f3b8-4e33-be0c-0454ad0c8d1e None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=b2396a5c6002491f926326ee2d154f92, role_id=7a6c27e9bb1d4ab0ae847254d4c8d5de)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:38.173759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e63888-f3b8-4e33-be0c-0454ad0c8d1e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:38.192795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 337/672] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:38 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/b2396a5c6002491f926326ee2d154f92/roles/7a6c27e9bb1d4ab0ae847254d4c8d5de => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:38.207680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31903ae-a5d3-40c2-aec0-00973d2d6fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:38.208318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31903ae-a5d3-40c2-aec0-00973d2d6fea None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:38.208603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31903ae-a5d3-40c2-aec0-00973d2d6fea None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:38.208882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31903ae-a5d3-40c2-aec0-00973d2d6fea None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:38.209562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31903ae-a5d3-40c2-aec0-00973d2d6fea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:38.210985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31903ae-a5d3-40c2-aec0-00973d2d6fea None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:38.231850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 336/673] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:00:38 2022] POST /identity/v3/services => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:39.287342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d118e-2332-43cb-9d41-a83903352229 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:39.287858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d118e-2332-43cb-9d41-a83903352229 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:39.287935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d118e-2332-43cb-9d41-a83903352229 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:39.289450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 338/674] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:39.295328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15071-bb78-4c3e-911f-268e02c68da0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:39.295744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15071-bb78-4c3e-911f-268e02c68da0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:39.295916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c15071-bb78-4c3e-911f-268e02c68da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:39.314607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a0c15071-bb78-4c3e-911f-268e02c68da0 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:39.315820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a0c15071-bb78-4c3e-911f-268e02c68da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:39.389434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 337/675] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:39.660825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d118e-2332-43cb-9d41-a83903352229 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:39.767790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d118e-2332-43cb-9d41-a83903352229 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:39.802951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:39.803611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:39.803896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:39.804173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:39.804759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:39.806159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:39.815323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:39.816307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:39.817255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 339/676] 10.222.0.42 () {56 vars in 1132 bytes} [Mon Oct 24 13:00:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:39.823444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c15071-bb78-4c3e-911f-268e02c68da0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:39.926103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:39.936442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c15071-bb78-4c3e-911f-268e02c68da0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:39.972703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:39.973313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:39.973600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:39.973875 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:39.974512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:39.974817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:39.976321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:39.982135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 338/677] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:39.988130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:40.038490 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:40.101337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843aa22d-0eae-478d-b611-6fc7cb6e4615 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:40.144061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44debe78-509a-4d10-a4e8-b71ddfcd68b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:40.144692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44debe78-509a-4d10-a4e8-b71ddfcd68b9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.144977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44debe78-509a-4d10-a4e8-b71ddfcd68b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.145254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44debe78-509a-4d10-a4e8-b71ddfcd68b9 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.145900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44debe78-509a-4d10-a4e8-b71ddfcd68b9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:40.146206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44debe78-509a-4d10-a4e8-b71ddfcd68b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:40.147628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44debe78-509a-4d10-a4e8-b71ddfcd68b9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:40.153717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 340/678] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:40.159395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:40.234649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1bebb-5e7e-4a5d-8db4-2e875aaec4fe None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.234948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1bebb-5e7e-4a5d-8db4-2e875aaec4fe None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.235133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1bebb-5e7e-4a5d-8db4-2e875aaec4fe None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.236793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 341/679] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:40.242187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326228-e640-42e8-98ee-f1ed7a2c6221 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.242745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326228-e640-42e8-98ee-f1ed7a2c6221 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.243258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b326228-e640-42e8-98ee-f1ed7a2c6221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.266105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0b326228-e640-42e8-98ee-f1ed7a2c6221 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:40.268343 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0b326228-e640-42e8-98ee-f1ed7a2c6221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:40.269486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843072c-705d-4515-a9c6-76a9b4d3d43c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:40.306773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf300e-bfda-4435-bbb6-50e004a65022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:40.307562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf300e-bfda-4435-bbb6-50e004a65022 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.307765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf300e-bfda-4435-bbb6-50e004a65022 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.308042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf300e-bfda-4435-bbb6-50e004a65022 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.308719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf300e-bfda-4435-bbb6-50e004a65022 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:40.310085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf300e-bfda-4435-bbb6-50e004a65022 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:40.341954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 339/680] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:00:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:40.365905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 342/681] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:40.503200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead65eb-8ba4-49f6-b849-3f4b214cb307 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.503385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead65eb-8ba4-49f6-b849-3f4b214cb307 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.503667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead65eb-8ba4-49f6-b849-3f4b214cb307 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.505268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 340/682] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:40.512082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372ace9-278a-4e55-b1fd-6e189b5cde74 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.512538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372ace9-278a-4e55-b1fd-6e189b5cde74 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.512894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372ace9-278a-4e55-b1fd-6e189b5cde74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.532324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a372ace9-278a-4e55-b1fd-6e189b5cde74 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:40.533543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a372ace9-278a-4e55-b1fd-6e189b5cde74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:40.546288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489110a1-a9d9-4de1-a37b-1e4258cc3294 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.546583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489110a1-a9d9-4de1-a37b-1e4258cc3294 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.546846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489110a1-a9d9-4de1-a37b-1e4258cc3294 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.548439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 341/683] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:40.553523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e573634-ad36-4e28-b7bc-d1695aa50946 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.553981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e573634-ad36-4e28-b7bc-d1695aa50946 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.554165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e573634-ad36-4e28-b7bc-d1695aa50946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.568619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:40.578852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8e573634-ad36-4e28-b7bc-d1695aa50946 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:40.580404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e573634-ad36-4e28-b7bc-d1695aa50946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:40.583427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:40.615363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 343/684] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:40.656817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372ace9-278a-4e55-b1fd-6e189b5cde74 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:40.662502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 342/685] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:40.766772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372ace9-278a-4e55-b1fd-6e189b5cde74 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:40.802830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:40.803502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.803782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.804055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.804636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:40.806259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:40.815326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:40.816279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:40.817134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 344/686] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:40.823117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e573634-ad36-4e28-b7bc-d1695aa50946 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:40.916519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:40.934255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e573634-ad36-4e28-b7bc-d1695aa50946 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:40.957651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:40.973137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:40.973748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:40.974030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:40.974306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:40.974942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:40.975289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:40.976656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:40.983168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 343/687] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:40.983909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:40.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:41.038312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe46e3-f644-4745-943e-c35619329d9a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.081055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfc000-96a6-4b18-93bd-68e27154dc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.081699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfc000-96a6-4b18-93bd-68e27154dc10 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.082416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfc000-96a6-4b18-93bd-68e27154dc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.082715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfc000-96a6-4b18-93bd-68e27154dc10 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.094540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfc000-96a6-4b18-93bd-68e27154dc10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:41.095994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d363e2f-54e6-41e1-9566-6b53be52631c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-dadfc000-96a6-4b18-93bd-68e27154dc10 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:41.099786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:41.100737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:00:41.101578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:41.106050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 345/688] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 24 13:00:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:41.106907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfc000-96a6-4b18-93bd-68e27154dc10 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.136385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.137021 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.137308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.137825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.138175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.139607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:41.148494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:41.149499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:41.150864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:41.155343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 344/689] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:41.156146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.205668 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:41.216418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfc000-96a6-4b18-93bd-68e27154dc10 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.231420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:41.252361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.252967 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.253250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.253526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.254152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:41.254458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.256236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.261995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 346/690] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:41.263013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.272560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a9cda-c79f-4ffd-afaa-e5b9d1f89c4e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.310261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.310906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.311262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.311605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.312340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:41.312714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.314148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.323438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 345/691] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:41.324175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.372200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d386a-4f32-45b4-ad18-dbce5ce71279 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.408904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.409832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.410265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.410554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.411238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:41.411550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.412919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.418747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 347/692] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:41.424002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.432455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ffad1-a6fa-4f3d-9d61-48d8bf1749ac None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.443538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:41.465728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:41.473675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.474279 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.474558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.474834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.475561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] RBAC: Adding query filter params (interface=public, service_id=d99dcbc3013f4b14ac14f6986d63e28f, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:41.475986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.477487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.487598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 346/693] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/endpoints?service_id=d99dcbc3013f4b14ac14f6986d63e28f&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:41.487598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.530982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b1ef8-a0a1-4eca-b824-3fbba2b5f525 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.564055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.564615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.564899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.565178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.576359 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.578269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.592726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab414f8a-563f-4cf2-bc6d-ff96f7d6762b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:41.592862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:41.593760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:41.599520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:41.599520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:41.599520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:41.599520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 348/694] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:41.599520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.638378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.638995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.639315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.639596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.640237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:41.640547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.641911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.646729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 347/695] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:41.647545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.675640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:41.712894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6dc2f5-e117-4be7-a435-3f65c46b4b58 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.714612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:41.748213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116021c-71f7-4baf-acee-d74509cce989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.748835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116021c-71f7-4baf-acee-d74509cce989 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.749121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116021c-71f7-4baf-acee-d74509cce989 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.749401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a116021c-71f7-4baf-acee-d74509cce989 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.750012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116021c-71f7-4baf-acee-d74509cce989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.751416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a116021c-71f7-4baf-acee-d74509cce989 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.757850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 349/696] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:00:41.758475 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116021c-71f7-4baf-acee-d74509cce989 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.774329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb6d2a-8f99-4a34-a8a1-be9752f27c39 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.823388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.823935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.824221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.824496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.825609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:41.825609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.827007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.841907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 348/697] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:41.850615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.895852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a116021c-71f7-4baf-acee-d74509cce989 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.935104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:41.935922 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:41.936200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:41.936475 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] PATH_INFO: `/v3/endpoints` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:41.937715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] RBAC: Adding query filter params (interface=public, service_id=9e0fa942b40243689517957473e7f8fb, region_id=RegionOne) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:41.938307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:41.939895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:41.942701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:41.952732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 350/698] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/endpoints?service_id=9e0fa942b40243689517957473e7f8fb&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:41.958690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:41.979226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7a15f-d2f0-489e-9a96-73c6ec20ffda None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:41.997990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:42.020383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:42.021143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:42.021540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:42.021938 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:42.029814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:42.031698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:42.045796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:42.046830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 24 13:00:42.047807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:42.048342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 349/699] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:42.054102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:42.092968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ddfa6c-0365-4d96-8510-7cfa4c52b9df None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:42.135258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a16dfc-e727-43ce-9327-a8cc39a03343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:42.136327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a16dfc-e727-43ce-9327-a8cc39a03343 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:42.136767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a16dfc-e727-43ce-9327-a8cc39a03343 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:42.137207 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a16dfc-e727-43ce-9327-a8cc39a03343 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:42.137965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a16dfc-e727-43ce-9327-a8cc39a03343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:42.139559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a16dfc-e727-43ce-9327-a8cc39a03343 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:42.146349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 351/700] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:00:41 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:00:42.192617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc6bf3-0c25-4f4c-bf29-e6f631e7fc73 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:42.232941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:42.233973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:42.234493 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:42.234982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:42.235968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:42.236477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:42.238141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:42.260242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 350/701] 10.222.0.42 () {56 vars in 1149 bytes} [Mon Oct 24 13:00:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:42.266300 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a16dfc-e727-43ce-9327-a8cc39a03343 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:42.341889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:42.385715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a16dfc-e727-43ce-9327-a8cc39a03343 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:42.422647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f65f1b-1f61-44df-b11e-d3efd17437da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:42.423296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f65f1b-1f61-44df-b11e-d3efd17437da None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:42.423579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f65f1b-1f61-44df-b11e-d3efd17437da None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:42.423851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f65f1b-1f61-44df-b11e-d3efd17437da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:42.497203 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f65f1b-1f61-44df-b11e-d3efd17437da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:42.498737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f65f1b-1f61-44df-b11e-d3efd17437da None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:42.571143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:42.616875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 352/702] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:00:42 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:00:42.623043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:42.638090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d374d8-5b8a-4bec-a147-f7483222532f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:42.638382 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d374d8-5b8a-4bec-a147-f7483222532f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:42.638651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d374d8-5b8a-4bec-a147-f7483222532f None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:42.643123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 353/703] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:42.645484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15d6da-ad53-442f-961a-cf5ed86168a6 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:42.645772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15d6da-ad53-442f-961a-cf5ed86168a6 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:42.646050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15d6da-ad53-442f-961a-cf5ed86168a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:42.666626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1c15d6da-ad53-442f-961a-cf5ed86168a6 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:42.667854 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1c15d6da-ad53-442f-961a-cf5ed86168a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:42.675254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:42.755081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 354/704] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:42.768900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb232932-cc0a-46e2-a6fb-141f4a5df48a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:42.805211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:42.806122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:42.806399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:42.806684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:42.817040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:42.818578 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:42.834184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:42.835155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:42.836073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:42.836073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:42.836073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:42.836073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 351/705] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:42.843007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15d6da-ad53-442f-961a-cf5ed86168a6 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:42.939400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:42.948824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15d6da-ad53-442f-961a-cf5ed86168a6 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:42.985227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:42.985851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:42.986134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:42.986408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:42.987069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:42.987386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:42.988760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:43.001282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 355/706] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:43.007099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:43.046753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:43.106581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:43.121983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce47d8-66c6-4059-a45e-a5f4f78187f4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:43.159732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:43.160377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:43.160659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:43.160932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:43.169107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3913fd-c7f7-4657-bdf8-10b8f247727c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:43.169790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:43.171415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:43.184312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:43.185256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:43.186128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:43.186128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:43.186128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:43.186128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:43.186128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:43.186128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:43.186128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:43.186128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:43.190481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 352/707] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:43.191714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:43.206059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:43.206813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:43.207140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:43.207480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:43.208139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:43.209527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:43.219248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:43.221440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:43.223411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 356/708] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:43.230077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:43.252728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:43.331883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd10c3-c11b-4c26-b3fb-79a2013841bb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:43.347710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97268e5d-7a4a-435d-ba15-4c9e35772c06 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:43.378735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:43.379829 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:43.380120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:43.380529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:43.381618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:43.381940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:43.383400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:43.386426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:43.387635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:43.388119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:43.388575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:43.389381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:43.389862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:43.391425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:43.398588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 353/709] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:43.398863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 357/710] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:43.404579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:43.408292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:43.417482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:43.513878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:43.535342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c41b0-b9f2-40f1-9e88-60e5a84c7f9f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:43.539292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cdcac2-54ea-4453-8fb2-54fefcdc495d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:43.573723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef0430-55f0-42ab-ab45-10f546fddd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:43.574347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0430-55f0-42ab-ab45-10f546fddd79 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:43.574636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0430-55f0-42ab-ab45-10f546fddd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:43.574911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef0430-55f0-42ab-ab45-10f546fddd79 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:43.575692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef0430-55f0-42ab-ab45-10f546fddd79 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:43.576003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef0430-55f0-42ab-ab45-10f546fddd79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:43.577468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef0430-55f0-42ab-ab45-10f546fddd79 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:43.586045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7338d00-5694-4dc8-a5c0-98907586afb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:43.587172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7338d00-5694-4dc8-a5c0-98907586afb1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:43.587669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7338d00-5694-4dc8-a5c0-98907586afb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:43.588247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7338d00-5694-4dc8-a5c0-98907586afb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:43.589245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7338d00-5694-4dc8-a5c0-98907586afb1 None admin] RBAC: Adding query filter params (role.id=7a6c27e9bb1d4ab0ae847254d4c8d5de, scope.project.id=00530732ebe84dababafb9f399452940, user.id=b2396a5c6002491f926326ee2d154f92) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:43.589792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7338d00-5694-4dc8-a5c0-98907586afb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:43.591092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 358/711] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:43.591753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7338d00-5694-4dc8-a5c0-98907586afb1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:43.599268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 354/712] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:00:43 2022] GET /identity/v3/role_assignments?user.id=b2396a5c6002491f926326ee2d154f92&scope.project.id=00530732ebe84dababafb9f399452940&role.id=7a6c27e9bb1d4ab0ae847254d4c8d5de => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:44.034808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a8882-1803-4aad-912a-39942aeadd2c None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:44.035235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a8882-1803-4aad-912a-39942aeadd2c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:44.035422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a8882-1803-4aad-912a-39942aeadd2c None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:44.036957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 359/713] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:44.043183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826311-eed0-41cd-92a7-bd85786d6444 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:44.044047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826311-eed0-41cd-92a7-bd85786d6444 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:44.044531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826311-eed0-41cd-92a7-bd85786d6444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:44.069694 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-21826311-eed0-41cd-92a7-bd85786d6444 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:44.077164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-21826311-eed0-41cd-92a7-bd85786d6444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:44.149232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 355/714] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:44.434733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a8882-1803-4aad-912a-39942aeadd2c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:44.525684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50ad3b-55b9-414f-83c8-5e0a1e265bf3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:44.526015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50ad3b-55b9-414f-83c8-5e0a1e265bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:44.526185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50ad3b-55b9-414f-83c8-5e0a1e265bf3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:44.527912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 356/715] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:44.533028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a5cb5-bb7a-47b2-bd0f-6cf6dd9d1b5c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:44.533316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a5cb5-bb7a-47b2-bd0f-6cf6dd9d1b5c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:44.533603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a5cb5-bb7a-47b2-bd0f-6cf6dd9d1b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:44.547078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:44.555378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-388a5cb5-bb7a-47b2-bd0f-6cf6dd9d1b5c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:44.557225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-388a5cb5-bb7a-47b2-bd0f-6cf6dd9d1b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:44.557822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a8882-1803-4aad-912a-39942aeadd2c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:44.597634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:44.598272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:44.598550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:44.598822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:44.599432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:44.600791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:44.610692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:44.611706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:44.612551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 360/716] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:44.618905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:44.632837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:44.640555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 357/717] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:44.737891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46101f39-e48b-48f7-a518-9fa3b14dd5e9 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:44.772462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:44.773117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:44.773409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:44.773681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:44.774309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:44.775155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:44.776592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:44.781671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 361/718] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:44.787347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a5cb5-bb7a-47b2-bd0f-6cf6dd9d1b5c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:44.893605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a5cb5-bb7a-47b2-bd0f-6cf6dd9d1b5c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:44.924527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:44.928199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:44.934303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:44.935203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:44.935668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:44.936164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:44.936970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:44.937456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:44.938987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:44.944726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 358/719] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:44.950680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:45.006836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:45.048618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8266d1-fc86-4aed-a490-611e02ea6d8c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:45.059505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7366a37-09f3-416e-9f0e-2c9692f8e898 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:45.101594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:45.102238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:45.103034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:45.103344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:45.104143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:45.105388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103c04d-98a3-4b9f-9be9-f3e7f339ff7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:45.105524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:45.106300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c04d-98a3-4b9f-9be9-f3e7f339ff7f None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:45.106761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c04d-98a3-4b9f-9be9-f3e7f339ff7f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:45.107370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103c04d-98a3-4b9f-9be9-f3e7f339ff7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:45.108363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103c04d-98a3-4b9f-9be9-f3e7f339ff7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:45.109901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1103c04d-98a3-4b9f-9be9-f3e7f339ff7f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:45.114431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:45.115450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:45.116290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 362/720] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:45.124130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:45.143169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 359/721] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:00:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:45.242273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7e0f8-bd37-417e-bba8-c3c0c07873a8 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:45.284652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:45.291759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:45.292821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:45.293119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:45.293455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:45.294114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:45.294419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:45.295904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:45.301386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 363/722] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:45.307781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103c04d-98a3-4b9f-9be9-f3e7f339ff7f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:45.371323 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:45.424565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1103c04d-98a3-4b9f-9be9-f3e7f339ff7f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:45.460956 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f25443-d65f-4462-b904-d4f4316242ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:45.461579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f25443-d65f-4462-b904-d4f4316242ff None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:45.462384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f25443-d65f-4462-b904-d4f4316242ff None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:45.462674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f25443-d65f-4462-b904-d4f4316242ff None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:45.463354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f25443-d65f-4462-b904-d4f4316242ff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:45.463663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f25443-d65f-4462-b904-d4f4316242ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:45.465033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f25443-d65f-4462-b904-d4f4316242ff None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:45.470542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 360/723] 10.222.0.42 () {56 vars in 1114 bytes} [Mon Oct 24 13:00:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:45.476176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:45.585308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ed2d0-0708-4cf5-962e-5b791d7619aa None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:45.622648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409829f-f8be-4017-8707-16c102feacba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:45.623308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409829f-f8be-4017-8707-16c102feacba None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:45.623593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409829f-f8be-4017-8707-16c102feacba None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:45.623869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409829f-f8be-4017-8707-16c102feacba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:45.624554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409829f-f8be-4017-8707-16c102feacba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:45.625924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409829f-f8be-4017-8707-16c102feacba None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:45.651629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 364/724] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:00:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:45.840684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51c1a-7284-472a-92c6-dd0ceeedcb26 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:45.840901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51c1a-7284-472a-92c6-dd0ceeedcb26 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:45.841187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c51c1a-7284-472a-92c6-dd0ceeedcb26 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:45.842778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 361/725] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:45.848435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3edd37-b6ce-42e3-90a0-28f52157ecee None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:45.848744 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3edd37-b6ce-42e3-90a0-28f52157ecee None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:45.849027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3edd37-b6ce-42e3-90a0-28f52157ecee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:45.870221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5f3edd37-b6ce-42e3-90a0-28f52157ecee None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:45.871529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5f3edd37-b6ce-42e3-90a0-28f52157ecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:45.928694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:45.957035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 365/726] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:46.001060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce390f-af34-4859-82fa-4baec13b6d93 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:46.001354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce390f-af34-4859-82fa-4baec13b6d93 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:46.001632 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce390f-af34-4859-82fa-4baec13b6d93 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:46.003237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 362/727] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:46.008731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5e3ed-c4e1-46dc-b19e-3e2f4fdd4e72 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:46.009505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5e3ed-c4e1-46dc-b19e-3e2f4fdd4e72 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:46.010102 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5e3ed-c4e1-46dc-b19e-3e2f4fdd4e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:46.028884 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0fb5e3ed-c4e1-46dc-b19e-3e2f4fdd4e72 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:46.030441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0fb5e3ed-c4e1-46dc-b19e-3e2f4fdd4e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:46.103245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 366/728] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:46.238145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce390f-af34-4859-82fa-4baec13b6d93 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:46.347739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce390f-af34-4859-82fa-4baec13b6d93 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:46.364892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:46.385154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5e3ed-c4e1-46dc-b19e-3e2f4fdd4e72 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:46.386042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d65a7-0263-4c9f-a621-3163a6257641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:46.386822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d65a7-0263-4c9f-a621-3163a6257641 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:46.387305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d65a7-0263-4c9f-a621-3163a6257641 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:46.387729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d65a7-0263-4c9f-a621-3163a6257641 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:46.388564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d65a7-0263-4c9f-a621-3163a6257641 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:46.390088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d65a7-0263-4c9f-a621-3163a6257641 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:46.414199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 363/729] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:00:46 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:46.507549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5e3ed-c4e1-46dc-b19e-3e2f4fdd4e72 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:46.519010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:46.555595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:46.556954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:46.557243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:46.557505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] PATH_INFO: `/v3/services/image` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:46.558099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:46.559537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:46.569528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:46.570660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:46.572125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 367/730] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Oct 24 13:00:46 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:46.577765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d65a7-0263-4c9f-a621-3163a6257641 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:46.690404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d65a7-0263-4c9f-a621-3163a6257641 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:46.726258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cdc9a-bef9-4ba5-8e49-dd2151eec11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:46.726869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cdc9a-bef9-4ba5-8e49-dd2151eec11f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:46.727178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cdc9a-bef9-4ba5-8e49-dd2151eec11f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:46.727983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cdc9a-bef9-4ba5-8e49-dd2151eec11f None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:46.728816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cdc9a-bef9-4ba5-8e49-dd2151eec11f None admin] RBAC: Adding query filter params (name=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:46.729124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cdc9a-bef9-4ba5-8e49-dd2151eec11f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:46.730483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cdc9a-bef9-4ba5-8e49-dd2151eec11f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:46.735623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 364/731] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:46.741217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:46.848508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f468a574-4865-411e-9ba5-8a9be7421f0e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:46.884574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af26e0-4881-4cc7-9d50-b8a3d22e0f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:46.885268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af26e0-4881-4cc7-9d50-b8a3d22e0f8a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:46.885557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af26e0-4881-4cc7-9d50-b8a3d22e0f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:46.885839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af26e0-4881-4cc7-9d50-b8a3d22e0f8a None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:46.886477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af26e0-4881-4cc7-9d50-b8a3d22e0f8a None admin] RBAC: Adding query filter params (type=image) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:46.886787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af26e0-4881-4cc7-9d50-b8a3d22e0f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:46.888294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af26e0-4881-4cc7-9d50-b8a3d22e0f8a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:46.893421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 368/732] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:47.522213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04900953-04f0-4ff5-aec5-3676ba3aab18 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:47.522736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04900953-04f0-4ff5-aec5-3676ba3aab18 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:47.522816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04900953-04f0-4ff5-aec5-3676ba3aab18 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:47.524278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 365/733] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:47.529732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7924d-7a61-4f01-b3cb-efb080e79922 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:47.530146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7924d-7a61-4f01-b3cb-efb080e79922 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:47.530315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7924d-7a61-4f01-b3cb-efb080e79922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:47.549306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e3a7924d-7a61-4f01-b3cb-efb080e79922 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:47.550834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e3a7924d-7a61-4f01-b3cb-efb080e79922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:47.579707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:47.624073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 369/734] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:47.893129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04900953-04f0-4ff5-aec5-3676ba3aab18 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:47.919432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c6b0-1303-479e-8da4-c6c8a4d94f2c None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:47.920037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c6b0-1303-479e-8da4-c6c8a4d94f2c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:47.920536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9002c6b0-1303-479e-8da4-c6c8a4d94f2c None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:47.921988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 370/735] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:47.927378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158aa2-4434-408b-9316-ede1a74842ef None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:47.927772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158aa2-4434-408b-9316-ede1a74842ef None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:47.928260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158aa2-4434-408b-9316-ede1a74842ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:47.940316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:47.949408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-51158aa2-4434-408b-9316-ede1a74842ef None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:47.951726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-51158aa2-4434-408b-9316-ede1a74842ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:48.010928 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04900953-04f0-4ff5-aec5-3676ba3aab18 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:48.043112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 371/736] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:48.055352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:48.056246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.056777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.057217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.057963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:48.059574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:48.070047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:48.071870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:48.072730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:48.075380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 366/737] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:48.086038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158aa2-4434-408b-9316-ede1a74842ef None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:48.183670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:48.192496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158aa2-4434-408b-9316-ede1a74842ef None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:48.228843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:48.229467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.229749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.230027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.230664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:48.230974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:48.232381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:48.237705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 372/738] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:48.243169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:48.345567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:48.350366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98421b9-bec7-430b-9c60-2bd54897b304 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:48.353575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:48.389034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:48.390026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.390458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.391020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.391823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:48.392274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:48.393802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:48.400599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 367/739] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:48.406199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:48.473791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7992cfca-bc53-4e20-bd79-bb4cd6e7254a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:48.512307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:48.512973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.513353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.513772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.514528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:48.515861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f076cc-a831-449b-bf1f-1d9a2dd07c37 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:48.516231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:48.519274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING keystone.server.flask.common [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 24 13:00:48.540186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.sql.core [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:00:48.540186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:00:48.540186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [parameters: {'id': '112606e211454b919df2f56adc9b5a68', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 24 13:00:48.540186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81614) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 112606e211454b919df2f56adc9b5a68.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 112606e211454b919df2f56adc9b5a68. Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:00:48.549856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result.read() Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 13:00:48.550876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application yield resource Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise self.value Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application session.commit() Oct 24 13:00:48.552065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:00:48.553196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:00:48.554318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result.read() Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:00:48.555319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [parameters: {'id': '112606e211454b919df2f56adc9b5a68', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:48.557100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:48.558084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 112606e211454b919df2f56adc9b5a68. Oct 24 13:00:48.559623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:48.563213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 373/740] 10.222.0.42 () {60 vars in 1140 bytes} [Mon Oct 24 13:00:48 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:48.566901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:48.568986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:48.569740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.570098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.570486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.572178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] RBAC: Adding query filter params (interface=admin, service_id=d99dcbc3013f4b14ac14f6986d63e28f, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:48.572566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:48.574929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:48.583232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 368/741] 10.222.0.42 () {56 vars in 1249 bytes} [Mon Oct 24 13:00:48 2022] GET /identity/v3/endpoints?service_id=d99dcbc3013f4b14ac14f6986d63e28f&interface=admin®ion_id=RegionOne => generated 193 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:48.587894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:48.591844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:48.628893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:48.720525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd21abf-5e84-450a-876e-4733517fb3fd None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:48.724965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fb99c-969d-4a74-b3b0-0e6f52ec862e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:48.764364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:48.765006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.765291 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.765567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.766947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e28fd-aa33-4b2a-9f85-6ce677a5f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:48.767910 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e28fd-aa33-4b2a-9f85-6ce677a5f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.768489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e28fd-aa33-4b2a-9f85-6ce677a5f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.768985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e28fd-aa33-4b2a-9f85-6ce677a5f2c1 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.769792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e28fd-aa33-4b2a-9f85-6ce677a5f2c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:48.771405 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e28fd-aa33-4b2a-9f85-6ce677a5f2c1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:48.772877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:48.775535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:48.780404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 369/742] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:00:48 2022] GET /identity/v3/services => generated 1727 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:48.789268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:48.790795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:48.791739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:48.791739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:48.791739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:48.791739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:48.791739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:48.791739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:48.791739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:48.791739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:48.794259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 374/743] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:48.803309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:48.803732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed6ca-cd28-4c44-b161-64dd254e4221 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.804034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed6ca-cd28-4c44-b161-64dd254e4221 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.804317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed6ca-cd28-4c44-b161-64dd254e4221 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.807169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 370/744] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:48.812644 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10ee39-fbb6-47ca-81d8-2bce1245f68d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.812994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10ee39-fbb6-47ca-81d8-2bce1245f68d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.813212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10ee39-fbb6-47ca-81d8-2bce1245f68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.832775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4b10ee39-fbb6-47ca-81d8-2bce1245f68d None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:48.833851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4b10ee39-fbb6-47ca-81d8-2bce1245f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:48.913697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:48.919467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 371/745] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:48.952676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742704a-2b7d-4dee-801c-a6176a1d1dca None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:48.990711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852b47e-f530-4a7b-b747-2276dd8ddea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:48.991750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852b47e-f530-4a7b-b747-2276dd8ddea0 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:48.992238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852b47e-f530-4a7b-b747-2276dd8ddea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:48.992707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852b47e-f530-4a7b-b747-2276dd8ddea0 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:48.994173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852b47e-f530-4a7b-b747-2276dd8ddea0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:48.994679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852b47e-f530-4a7b-b747-2276dd8ddea0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:48.996432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852b47e-f530-4a7b-b747-2276dd8ddea0 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:49.018624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 375/746] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:49.161450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c28b3-19a1-4d33-a9ba-c0770bccbda6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:49.161796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c28b3-19a1-4d33-a9ba-c0770bccbda6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:49.162016 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c28b3-19a1-4d33-a9ba-c0770bccbda6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:49.163746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 372/747] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:49.169620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f7fae-b53b-42de-ae8b-1fc85e3852df None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:49.169942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f7fae-b53b-42de-ae8b-1fc85e3852df None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:49.170194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f7fae-b53b-42de-ae8b-1fc85e3852df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:49.188583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c28b3-19a1-4d33-a9ba-c0770bccbda6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:49.191512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0f0f7fae-b53b-42de-ae8b-1fc85e3852df None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:49.193071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0f0f7fae-b53b-42de-ae8b-1fc85e3852df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:49.222157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:49.280667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 376/748] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:49.314270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c28b3-19a1-4d33-a9ba-c0770bccbda6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:49.349898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:49.350537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:49.350818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:49.351117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] PATH_INFO: `/v3/services/compute` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:49.351768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:49.353922 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:49.363076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:49.364147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:49.364983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 373/749] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:49.372315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f7fae-b53b-42de-ae8b-1fc85e3852df None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:49.437885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:49.484050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f7fae-b53b-42de-ae8b-1fc85e3852df None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:49.520785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:49.521761 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:49.522291 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:49.522765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:49.523624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:49.524132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:49.526262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:49.532180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 377/750] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:49.540374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:49.558730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:49.652160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:49.667794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5306886-a854-4af6-8120-9dbdcf3f7cdd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:49.687338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97ecd6-8e57-4535-acaf-1471f7ea8194 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:49.706436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:49.707074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:49.707489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:49.707785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:49.708697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:49.709010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:49.710803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:49.721971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 374/751] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:49.722659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:49.723639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:49.739821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a561db7-9096-4043-a5fe-437af54a3585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:49.741384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a561db7-9096-4043-a5fe-437af54a3585 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:49.741929 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a561db7-9096-4043-a5fe-437af54a3585 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:49.742371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a561db7-9096-4043-a5fe-437af54a3585 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:49.743740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a561db7-9096-4043-a5fe-437af54a3585 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:49.746170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a561db7-9096-4043-a5fe-437af54a3585 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:49.769038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 378/752] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:00:49 2022] POST /identity/v3/services => generated 242 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:00:49.849145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b63700-bed4-4b28-84b2-e4d532753399 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:49.896605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1386da1-8684-4314-891a-022bf9b49b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:49.897242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1386da1-8684-4314-891a-022bf9b49b3c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:49.897527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1386da1-8684-4314-891a-022bf9b49b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:49.897798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1386da1-8684-4314-891a-022bf9b49b3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:49.898521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1386da1-8684-4314-891a-022bf9b49b3c None admin] RBAC: Adding query filter params (interface=public, service_id=e9fde2a8ead0455db8bb26a7cd6667f6, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:49.898891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1386da1-8684-4314-891a-022bf9b49b3c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:49.900965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1386da1-8684-4314-891a-022bf9b49b3c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:49.907048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 375/753] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 24 13:00:49 2022] GET /identity/v3/endpoints?service_id=e9fde2a8ead0455db8bb26a7cd6667f6&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:49.913096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a561db7-9096-4043-a5fe-437af54a3585 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:50.030924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a561db7-9096-4043-a5fe-437af54a3585 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:50.078200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe5b164-570d-4578-a8e5-e453449ff0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:50.078834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5b164-570d-4578-a8e5-e453449ff0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:50.079143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5b164-570d-4578-a8e5-e453449ff0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:50.079418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe5b164-570d-4578-a8e5-e453449ff0e9 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:50.080031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5b164-570d-4578-a8e5-e453449ff0e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:50.081399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe5b164-570d-4578-a8e5-e453449ff0e9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:50.088990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 379/754] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:00:49 2022] GET /identity/v3/services => generated 1957 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:00:51.049684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5791ec6-f45c-45cc-9d10-3161c4dce2fa None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:51.050206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5791ec6-f45c-45cc-9d10-3161c4dce2fa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:51.050511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5791ec6-f45c-45cc-9d10-3161c4dce2fa None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:51.052230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 376/755] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:51.058271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bfd8a-f6a4-44de-a7b5-cb91aa215162 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:51.058675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bfd8a-f6a4-44de-a7b5-cb91aa215162 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:51.059017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bfd8a-f6a4-44de-a7b5-cb91aa215162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:51.089484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-840bfd8a-f6a4-44de-a7b5-cb91aa215162 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:51.090685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-840bfd8a-f6a4-44de-a7b5-cb91aa215162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:51.139297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:51.162489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 380/756] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:51 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:51.403194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba596bf-0abe-4bf7-afc9-9afc2c6b18fb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:51.403501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba596bf-0abe-4bf7-afc9-9afc2c6b18fb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:51.403861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba596bf-0abe-4bf7-afc9-9afc2c6b18fb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:51.405457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 377/757] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:51.411248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52500c0-afca-4eb9-9f68-cb65442582a5 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:51.411545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52500c0-afca-4eb9-9f68-cb65442582a5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:51.411831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52500c0-afca-4eb9-9f68-cb65442582a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:51.430596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e52500c0-afca-4eb9-9f68-cb65442582a5 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:51.431830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e52500c0-afca-4eb9-9f68-cb65442582a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:51.438129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba596bf-0abe-4bf7-afc9-9afc2c6b18fb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:51.454449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:51.507402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 381/758] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:51 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:51.546934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba596bf-0abe-4bf7-afc9-9afc2c6b18fb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:51.581532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:51.582423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:51.582855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:51.583305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:51.584033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:51.585527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:51.595246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:51.596425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 24 13:00:51.597466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:51.599881 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 378/759] 10.222.0.42 () {56 vars in 1126 bytes} [Mon Oct 24 13:00:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:51.606544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52500c0-afca-4eb9-9f68-cb65442582a5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:51.713083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52500c0-afca-4eb9-9f68-cb65442582a5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:51.724282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:51.749960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:51.750578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:51.750864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:51.751170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:51.751811 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:51.752122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:51.753497 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:51.759968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 382/760] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:51.765466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:51.786379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:51.855449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:51.879590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55cde6-f2da-454f-92c5-01c6158edd16 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:51.916624 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:51.917244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:51.917527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:51.917804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:51.918448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:51.918759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:51.920172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:51.920984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b84a15-f7ce-43f9-817b-afaa52f8ea56 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:51.930106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 379/761] 10.222.0.42 () {56 vars in 1122 bytes} [Mon Oct 24 13:00:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:51.931684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:51.967755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:51.968295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:51.968582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:51.968860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:51.969956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:51.971949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:51.990850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.sql.core [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:00:51.990850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 24 13:00:51.990850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [parameters: {'user_id': 'ac18cc28c2dc4179b5a7abfb1039625d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 24 13:00:51.990850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81614) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result.read() Oct 24 13:00:51.998405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:00:51.999454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application yield resource Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise self.value Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application session.commit() Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:00:52.000390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:00:52.001275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:00:52.002159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application result.read() Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:00:52.003015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac18cc28c2dc4179b5a7abfb1039625d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:52.003869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 24 13:00:52.005057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 24 13:00:52.008092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 24 13:00:52.008092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 24 13:00:52.008092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:00:52.008092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:00:52.008092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 24 13:00:52.008092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:52.011143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 383/762] 10.222.0.42 () {60 vars in 1131 bytes} [Mon Oct 24 13:00:51 2022] POST /identity/v3/users => generated 160 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:52.017423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:52.064180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:52.075859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b348b13-235d-4848-9a54-4e1fd0ba7bbd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:52.115730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a9651-15e7-4a47-b7bc-bbf2b4f6bf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:52.116436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9651-15e7-4a47-b7bc-bbf2b4f6bf3a None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.116790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9651-15e7-4a47-b7bc-bbf2b4f6bf3a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.117136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9651-15e7-4a47-b7bc-bbf2b4f6bf3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.118113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a9651-15e7-4a47-b7bc-bbf2b4f6bf3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:52.119558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a9651-15e7-4a47-b7bc-bbf2b4f6bf3a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:52.145877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 380/763] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:00:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:52.172774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0815-928e-47bc-88bc-95952f74e6ea None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:52.179068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7d242-4f2e-44f9-8a56-7be1fee5360e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.179412 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7d242-4f2e-44f9-8a56-7be1fee5360e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.179656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7d242-4f2e-44f9-8a56-7be1fee5360e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.182706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 381/764] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:52.188957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c1ba9-8a6e-4f56-a24c-f91de83eafb5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.189305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c1ba9-8a6e-4f56-a24c-f91de83eafb5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.189514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c1ba9-8a6e-4f56-a24c-f91de83eafb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.196413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:52.216500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2c2c1ba9-8a6e-4f56-a24c-f91de83eafb5 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:52.217997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2c2c1ba9-8a6e-4f56-a24c-f91de83eafb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:52.242424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:52.243783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.244073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.244340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.257282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:52.259361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:52.276445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:52.278437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:52.280001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 384/765] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:00:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:52.284846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:52.311237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 382/766] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:52 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:52.312672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ffa3-4bfe-481f-9103-7c0138804e4f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.313618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ffa3-4bfe-481f-9103-7c0138804e4f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.313911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ffa3-4bfe-481f-9103-7c0138804e4f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.319289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 383/767] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:52.322183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502243f-9e19-48eb-9461-fd1cfab0743c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.322558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502243f-9e19-48eb-9461-fd1cfab0743c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.322828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502243f-9e19-48eb-9461-fd1cfab0743c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.343238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9502243f-9e19-48eb-9461-fd1cfab0743c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:52.344917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9502243f-9e19-48eb-9461-fd1cfab0743c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:52.408149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:52.417553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98772d9-b8f0-48cf-991d-da40a3dd559c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:52.437470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 384/768] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:52 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:52.460396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ece02-ee32-490f-b362-70767663e914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:52.461276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ece02-ee32-490f-b362-70767663e914 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.461770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ece02-ee32-490f-b362-70767663e914 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.462194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ece02-ee32-490f-b362-70767663e914 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.463043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ece02-ee32-490f-b362-70767663e914 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:52.463545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ece02-ee32-490f-b362-70767663e914 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:52.465131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ece02-ee32-490f-b362-70767663e914 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:52.507227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 385/769] 10.222.0.42 () {56 vars in 1101 bytes} [Mon Oct 24 13:00:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:52.591920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502243f-9e19-48eb-9461-fd1cfab0743c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:52.662249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:52.711433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502243f-9e19-48eb-9461-fd1cfab0743c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:52.715195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ece02-ee32-490f-b362-70767663e914 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:52.757105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:52.757360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.757574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.757849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] PATH_INFO: `/v3/services/image` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.758962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:52.760642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:52.772827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:52.773947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:52.775824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 385/770] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Oct 24 13:00:52 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:52.781026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:52.855003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ece02-ee32-490f-b362-70767663e914 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:52.890826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:52.891492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.891785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.892068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.892723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:52.894106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:52.903039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:52.904043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:52.904901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 386/771] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:52.909966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:52.912977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf299-5eec-44ae-b9c2-bc3b41bd4c08 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:52.925883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:52.944197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:52.954895 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:52.955887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:52.956175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:52.956771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:52.957418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] RBAC: Adding query filter params (name=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:52.957730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:52.959159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:52.969474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 386/772] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:52.970884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:53.021503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73bab1-0ea4-40d1-9059-2e0205a5889f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:53.059717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:53.060337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:53.060741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:53.061022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:53.063092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:53.063092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:53.063515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:53.070552 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 387/773] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:53.076440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:53.083930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b6234-8daa-46d9-bc55-d94979902593 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:53.122354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:53.122974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:53.123607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:53.123890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:53.124681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] RBAC: Adding query filter params (type=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:53.124999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:53.126367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:53.137081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 387/774] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:53.138188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:53.170138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:53.193998 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad3222-fa0b-436a-b2b3-c5f6487144c9 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:53.199978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:53.231591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:53.232312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:53.232601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:53.232880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:53.233511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:53.233821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:53.235335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:53.240642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 388/775] 10.222.0.42 () {56 vars in 1112 bytes} [Mon Oct 24 13:00:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:53.246591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:53.247076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1bd2f-cdd3-42f8-a578-99e3212446aa None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:53.282985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:53.283552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:53.283840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:53.284117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:53.284816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] RBAC: Adding query filter params (interface=public, service_id=019bd22e08f74156b2f8a446886a01f1, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:53.285129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:53.286486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:53.292096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 388/776] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 24 13:00:53 2022] GET /identity/v3/endpoints?service_id=019bd22e08f74156b2f8a446886a01f1&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:53.297629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:53.358764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db1d63-e1ad-40fc-b1e1-9f649fa3e1f6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:53.393916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6956e23-fa87-469a-a888-8602a8d25e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:53.394529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956e23-fa87-469a-a888-8602a8d25e69 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:53.394816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956e23-fa87-469a-a888-8602a8d25e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:53.395123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956e23-fa87-469a-a888-8602a8d25e69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:53.395787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6956e23-fa87-469a-a888-8602a8d25e69 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:53.397157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6956e23-fa87-469a-a888-8602a8d25e69 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:53.402947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2ba71f-123f-4e3e-9287-c38c4a97dddb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:53.415329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:53.421434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:53.424110 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 389/777] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:00:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:00:53.453212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e972a-3b81-4274-b539-6ea2b80500f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:53.453862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e972a-3b81-4274-b539-6ea2b80500f4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:53.454136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e972a-3b81-4274-b539-6ea2b80500f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:53.454437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e972a-3b81-4274-b539-6ea2b80500f4 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:53.455083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e972a-3b81-4274-b539-6ea2b80500f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:53.456570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e972a-3b81-4274-b539-6ea2b80500f4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:53.471328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 389/778] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:00:53 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:00:54.464451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7662533-1fc5-4dff-a52f-98fb7552063f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:54.464451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7662533-1fc5-4dff-a52f-98fb7552063f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:54.464451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7662533-1fc5-4dff-a52f-98fb7552063f None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:54.467096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 390/779] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:54.470673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689b1c6-28d0-42ab-b49a-60f57a6166e1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:54.471090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689b1c6-28d0-42ab-b49a-60f57a6166e1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:54.471276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689b1c6-28d0-42ab-b49a-60f57a6166e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:54.489223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f689b1c6-28d0-42ab-b49a-60f57a6166e1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:54.490391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f689b1c6-28d0-42ab-b49a-60f57a6166e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:54.559169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 390/780] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:54.833834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7662533-1fc5-4dff-a52f-98fb7552063f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:54.945677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7662533-1fc5-4dff-a52f-98fb7552063f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:54.979656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90554941-9ee8-4c03-a914-5a25794328b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:54.980283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90554941-9ee8-4c03-a914-5a25794328b6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:54.980568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90554941-9ee8-4c03-a914-5a25794328b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:54.980843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90554941-9ee8-4c03-a914-5a25794328b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:54.981949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90554941-9ee8-4c03-a914-5a25794328b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:54.983397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90554941-9ee8-4c03-a914-5a25794328b6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:54.985116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 391/781] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:54.990728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689b1c6-28d0-42ab-b49a-60f57a6166e1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:55.098682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689b1c6-28d0-42ab-b49a-60f57a6166e1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:55.108482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:55.134965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcda71d-9fc7-4917-a27c-5eca674c0940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:55.135600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda71d-9fc7-4917-a27c-5eca674c0940 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:55.135882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda71d-9fc7-4917-a27c-5eca674c0940 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:55.136153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcda71d-9fc7-4917-a27c-5eca674c0940 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:55.136789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda71d-9fc7-4917-a27c-5eca674c0940 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:55.138207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcda71d-9fc7-4917-a27c-5eca674c0940 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:55.166619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 391/782] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:00:54 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:00:55.663463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a629e3-2d51-4f92-a218-864c49d91711 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:55.663984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a629e3-2d51-4f92-a218-864c49d91711 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:55.664062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a629e3-2d51-4f92-a218-864c49d91711 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:55.666042 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 392/783] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:55.671204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb1b5e-9454-40c3-95f1-404ede50b3aa None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:55.671639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb1b5e-9454-40c3-95f1-404ede50b3aa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:55.671813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb1b5e-9454-40c3-95f1-404ede50b3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:55.690559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f1cb1b5e-9454-40c3-95f1-404ede50b3aa None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:55.691785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f1cb1b5e-9454-40c3-95f1-404ede50b3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:55.702788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebc19e-ab70-410b-89b3-5a82415b3870 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:55.703165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebc19e-ab70-410b-89b3-5a82415b3870 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:55.703387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebc19e-ab70-410b-89b3-5a82415b3870 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:55.704854 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 393/784] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:00:55.709549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2049a5-03be-4195-92e1-c0c04bd7423a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:55.709839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2049a5-03be-4195-92e1-c0c04bd7423a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:55.710121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2049a5-03be-4195-92e1-c0c04bd7423a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:55.729516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6d2049a5-03be-4195-92e1-c0c04bd7423a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:55.731039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6d2049a5-03be-4195-92e1-c0c04bd7423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:55.764823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 392/785] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:55.792828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:55.801943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 394/786] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:56.040833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb1b5e-9454-40c3-95f1-404ede50b3aa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.075899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2049a5-03be-4195-92e1-c0c04bd7423a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.145800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:56.158200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb1b5e-9454-40c3-95f1-404ede50b3aa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.182530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2049a5-03be-4195-92e1-c0c04bd7423a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.196621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.197261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.197541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.197819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] PATH_INFO: `/v3/services/image` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.198401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:56.199788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:56.208474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:56.209463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:56.210297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 393/787] 10.222.0.42 () {56 vars in 1105 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:56.216480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.220117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a95a45-2f2e-4ae8-bea5-31236b23afdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.220751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a95a45-2f2e-4ae8-bea5-31236b23afdd None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.221028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a95a45-2f2e-4ae8-bea5-31236b23afdd None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.221299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a95a45-2f2e-4ae8-bea5-31236b23afdd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.231538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a95a45-2f2e-4ae8-bea5-31236b23afdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-c3a95a45-2f2e-4ae8-bea5-31236b23afdd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:00:56.236400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:56.237328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:56.238868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 395/788] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:56.244151 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a95a45-2f2e-4ae8-bea5-31236b23afdd None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.303040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:56.324570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ff115-aa32-4a77-9ff3-ecf587588695 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.347879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a95a45-2f2e-4ae8-bea5-31236b23afdd None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.359761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.360450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.360741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.361018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.361650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] RBAC: Adding query filter params (name=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:56.361957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:56.363364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:56.368467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 394/789] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:56.373852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.384827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.385429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.385713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.385985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.386644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:56.386946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:56.388347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:56.394079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:56.397114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 396/790] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:56.403079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.481274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4319a-5551-444b-b41c-0d923386da55 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.509932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebe64b-740c-4ae5-a937-42ae4dfe4f8e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.516914 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.517526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.517813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.518088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.518714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] RBAC: Adding query filter params (type=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:56.519027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:56.520407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:56.526311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 395/791] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:56.532383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:56.532583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.552211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.552843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.553178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.553405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.564298 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:56.565762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:56.581731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:56.582695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:56.584371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:56.584371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:56.584371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:56.584371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 397/792] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:56.589646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.622203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:56.644286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e8182-6f31-473d-8fb8-7f2e90970da5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.677494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de89b6-f8e6-41ee-b2d4-19d833b62055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.678121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de89b6-f8e6-41ee-b2d4-19d833b62055 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.678375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de89b6-f8e6-41ee-b2d4-19d833b62055 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.678653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de89b6-f8e6-41ee-b2d4-19d833b62055 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.679370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de89b6-f8e6-41ee-b2d4-19d833b62055 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:56.681559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67de89b6-f8e6-41ee-b2d4-19d833b62055 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:56.703263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 396/793] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:00:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:00:56.703625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352e353-890d-4d21-a169-3d36ef76d38d None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.750780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.751719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.752141 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.752556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.753349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:56.753818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:56.755461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:56.775714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 398/794] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:56.777251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de89b6-f8e6-41ee-b2d4-19d833b62055 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.893421 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de89b6-f8e6-41ee-b2d4-19d833b62055 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:56.928317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:56.928959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:56.929242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:56.929521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:56.937783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:56.939265 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:56.941081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:00:56.952440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:56.953719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:56.954863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 397/795] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:56.960174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:56.987896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:57.067892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0591e0-ccfc-4b05-8eb6-d7994bedea2a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:57.104174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:57.104792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:57.105078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:57.105355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:57.106285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:57.106602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:57.108127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:57.131692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 399/796] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 24 13:00:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:57.133320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:57.237634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c1adc4-5e46-48da-a3f1-80d6abd9fd47 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:57.271822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81aac87-a433-4f64-8626-2b09976fb57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:57.272477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aac87-a433-4f64-8626-2b09976fb57d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:57.272761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aac87-a433-4f64-8626-2b09976fb57d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:57.273038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81aac87-a433-4f64-8626-2b09976fb57d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:57.285480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81aac87-a433-4f64-8626-2b09976fb57d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:57.287014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81aac87-a433-4f64-8626-2b09976fb57d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a81aac87-a433-4f64-8626-2b09976fb57d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:57.303417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:57.304435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:57.305990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:57.305990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:57.305990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:57.307124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 398/797] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:57.310898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:57.392590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ad617-361e-4eb2-9b95-193b44259b68 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:57.392917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ad617-361e-4eb2-9b95-193b44259b68 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:57.393137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ad617-361e-4eb2-9b95-193b44259b68 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:57.394680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 399/798] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:57.399985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334cba5-c8ad-4917-878d-8919cc3c8081 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:57.400285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334cba5-c8ad-4917-878d-8919cc3c8081 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:57.400541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334cba5-c8ad-4917-878d-8919cc3c8081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:57.401148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:57.420126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3334cba5-c8ad-4917-878d-8919cc3c8081 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:57.421292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3334cba5-c8ad-4917-878d-8919cc3c8081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:57.421891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c3578-2a9c-49cb-95ba-57f7e925ec65 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:57.462734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:57.463316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:57.463600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:57.463874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:57.464534 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:57.464842 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:57.466268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:57.485192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 400/799] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:57.486494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:57.589823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:57.603709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99063dc5-0c02-40a2-b559-cdfa380ed40a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:57.613731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 400/800] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:00:57.639712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:57.640366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:57.640644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:57.640918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:57.648499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:57.651257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:00:57.662752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:57.663800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:57.665432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:00:57.665432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:00:57.665432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:00:57.665432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:00:57.665432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:00:57.665432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:00:57.665432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:00:57.665432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:57.667361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 401/801] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:00:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:00:57.670697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334cba5-c8ad-4917-878d-8919cc3c8081 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:57.772736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334cba5-c8ad-4917-878d-8919cc3c8081 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:57.807607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:57.808207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:57.808484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:57.808754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:57.809436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:57.809742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:57.811184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:57.823611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 401/802] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:00:57.829290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:57.834177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:57.890781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:57.935935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:57.943531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f3147-f39c-4146-b2a1-ec9095ba44d2 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:57.980977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667fa86-1555-44e1-92b2-618b9547395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:57.981938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667fa86-1555-44e1-92b2-618b9547395b None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:57.982424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667fa86-1555-44e1-92b2-618b9547395b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:57.982704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f667fa86-1555-44e1-92b2-618b9547395b None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/2dff78b5eadc42608afbf2b14dd07e7a/roles/7a6c27e9bb1d4ab0ae847254d4c8d5de` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:57.984884 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667fa86-1555-44e1-92b2-618b9547395b None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=2dff78b5eadc42608afbf2b14dd07e7a, role_id=7a6c27e9bb1d4ab0ae847254d4c8d5de)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:57.986330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f667fa86-1555-44e1-92b2-618b9547395b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:58.001131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb00ea-a942-477e-a82e-5e137e2ba0bf None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:58.002258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 402/803] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:00:57 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/2dff78b5eadc42608afbf2b14dd07e7a/roles/7a6c27e9bb1d4ab0ae847254d4c8d5de => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:00:58.047166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cadf3b-ee99-4f3e-87a0-7d9ee3eeabaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:58.047876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cadf3b-ee99-4f3e-87a0-7d9ee3eeabaa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:58.048161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cadf3b-ee99-4f3e-87a0-7d9ee3eeabaa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:58.048435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cadf3b-ee99-4f3e-87a0-7d9ee3eeabaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:58.049576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cadf3b-ee99-4f3e-87a0-7d9ee3eeabaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:58.051127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cadf3b-ee99-4f3e-87a0-7d9ee3eeabaa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:58.052934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 402/804] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:00:58.059593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667fa86-1555-44e1-92b2-618b9547395b None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:58.138047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:58.182328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f667fa86-1555-44e1-92b2-618b9547395b None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:58.216620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60a5f5-8ab1-477c-bc70-1634a4130ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:58.217236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60a5f5-8ab1-477c-bc70-1634a4130ff6 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:58.217520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60a5f5-8ab1-477c-bc70-1634a4130ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:58.217794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60a5f5-8ab1-477c-bc70-1634a4130ff6 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:58.218455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60a5f5-8ab1-477c-bc70-1634a4130ff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:58.219914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60a5f5-8ab1-477c-bc70-1634a4130ff6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:58.241777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 403/805] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:00:58 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:00:58.963882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c772ff-f7ee-401c-98f2-38cd59c9a9df None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:58.964425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c772ff-f7ee-401c-98f2-38cd59c9a9df None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:58.964425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c772ff-f7ee-401c-98f2-38cd59c9a9df None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:58.966165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 403/806] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:00:58.972133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a53a4f-608d-4f61-a7c6-816261d407ec None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:58.972675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a53a4f-608d-4f61-a7c6-816261d407ec None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:58.972675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a53a4f-608d-4f61-a7c6-816261d407ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:58.994063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f2a53a4f-608d-4f61-a7c6-816261d407ec None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:00:58.995310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f2a53a4f-608d-4f61-a7c6-816261d407ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:00:59.083097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 404/807] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:00:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:00:59.398271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c772ff-f7ee-401c-98f2-38cd59c9a9df None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:59.532470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c772ff-f7ee-401c-98f2-38cd59c9a9df None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:59.542548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:59.567654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:59.568266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:59.568555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:59.568833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:59.569500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:59.571380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:59.580130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.sql.core [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:00:59.580130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:00:59.580130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [parameters: {'id': 'c4bd1f8e888643a4bf5cd35c214c4110', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:00:59.580130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81613) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:00:59.587980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 24 13:00:59.588846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application yield resource Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.rollback() Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.force_reraise() Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise self.value Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application session.commit() Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 24 13:00:59.589854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._prepare_impl() Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.flush() Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._flush(objects) Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application compat.raise_( Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application flush_context.execute() Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rec.execute(self) Oct 24 13:00:59.590772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 24 13:00:59.591743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 24 13:00:59.593402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [parameters: {'id': 'c4bd1f8e888643a4bf5cd35c214c4110', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 24 13:00:59.599808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:59.606059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:00:59.607369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 404/808] 10.222.0.42 () {60 vars in 1137 bytes} [Mon Oct 24 13:00:59 2022] POST /identity/v3/domains => generated 190 bytes in 196 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:59.608262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a53a4f-608d-4f61-a7c6-816261d407ec None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:59.653027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:00:59.708484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a53a4f-608d-4f61-a7c6-816261d407ec None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:59.744029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f1897-8913-4eca-afb5-144c91a243eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:59.744944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f1897-8913-4eca-afb5-144c91a243eb None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:59.745385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f1897-8913-4eca-afb5-144c91a243eb None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:59.745807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f1897-8913-4eca-afb5-144c91a243eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:59.756306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f1897-8913-4eca-afb5-144c91a243eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:59.757912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f1897-8913-4eca-afb5-144c91a243eb None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-5c7f1897-8913-4eca-afb5-144c91a243eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:00:59.773527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:00:59.774624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:00:59.775648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:00:59.775648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:00:59.775648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:00:59.775648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 405/809] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:00:59.781447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:00:59.886330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ece0f7-4431-404d-af0c-a604cc9750d4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:00:59.921251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9886f5bb-a1ae-4b2f-96fb-3c6e03e1a856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:00:59.921788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886f5bb-a1ae-4b2f-96fb-3c6e03e1a856 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:00:59.922072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886f5bb-a1ae-4b2f-96fb-3c6e03e1a856 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:00:59.922352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9886f5bb-a1ae-4b2f-96fb-3c6e03e1a856 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:00:59.923002 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886f5bb-a1ae-4b2f-96fb-3c6e03e1a856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:00:59.923337 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886f5bb-a1ae-4b2f-96fb-3c6e03e1a856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:00:59.924702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9886f5bb-a1ae-4b2f-96fb-3c6e03e1a856 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:00:59.936375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 405/810] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:00.347613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded7bed-e547-4a81-9437-38bdf7d61fe2 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:00.348136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded7bed-e547-4a81-9437-38bdf7d61fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:00.348136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded7bed-e547-4a81-9437-38bdf7d61fe2 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:00.349422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 406/811] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:00.354726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7f39c-8659-47de-a2cb-d316f75bde29 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:00.355187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7f39c-8659-47de-a2cb-d316f75bde29 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:00.355298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b7f39c-8659-47de-a2cb-d316f75bde29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:00.372834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a2b7f39c-8659-47de-a2cb-d316f75bde29 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:00.373998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a2b7f39c-8659-47de-a2cb-d316f75bde29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:00.402232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:00.446705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 406/812] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:00.509828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55c1f9-7d2a-4f1a-8691-3de00f924c51 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:00.510101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55c1f9-7d2a-4f1a-8691-3de00f924c51 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:00.510373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55c1f9-7d2a-4f1a-8691-3de00f924c51 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:00.512134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 407/813] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:00.517821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030db3-5707-4308-ab88-f9daf8518ffe None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:00.518109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030db3-5707-4308-ab88-f9daf8518ffe None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:00.518385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030db3-5707-4308-ab88-f9daf8518ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:00.536643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-74030db3-5707-4308-ab88-f9daf8518ffe None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:00.537805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-74030db3-5707-4308-ab88-f9daf8518ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:00.605421 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 407/814] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 24 13:01:00.716903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55c1f9-7d2a-4f1a-8691-3de00f924c51 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:00.881654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74030db3-5707-4308-ab88-f9daf8518ffe None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:00.936023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55c1f9-7d2a-4f1a-8691-3de00f924c51 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:00.937562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:00.970677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7697068-78d3-4bd9-a227-237bd9ae4a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:00.971336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7697068-78d3-4bd9-a227-237bd9ae4a50 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:00.971621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7697068-78d3-4bd9-a227-237bd9ae4a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:00.971905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7697068-78d3-4bd9-a227-237bd9ae4a50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:00.972848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:00.982560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74030db3-5707-4308-ab88-f9daf8518ffe None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:00.982832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7697068-78d3-4bd9-a227-237bd9ae4a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-a7697068-78d3-4bd9-a227-237bd9ae4a50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:01:00.988025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:00.989206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:01:00.990314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 408/815] 10.222.0.42 () {56 vars in 1120 bytes} [Mon Oct 24 13:01:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:01:00.995703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7697068-78d3-4bd9-a227-237bd9ae4a50 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.018880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dc8bf-c531-4a9f-a603-2794bae87a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.019550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dc8bf-c531-4a9f-a603-2794bae87a7c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.019828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dc8bf-c531-4a9f-a603-2794bae87a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.020102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450dc8bf-c531-4a9f-a603-2794bae87a7c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.030312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450dc8bf-c531-4a9f-a603-2794bae87a7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-450dc8bf-c531-4a9f-a603-2794bae87a7c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:01:01.035159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:01.036287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:01:01.037136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 408/816] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:01:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:01:01.042652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dc8bf-c531-4a9f-a603-2794bae87a7c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.100188 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7697068-78d3-4bd9-a227-237bd9ae4a50 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.134687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.135265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.135547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.135819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.136484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:01.136787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.138161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.142140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450dc8bf-c531-4a9f-a603-2794bae87a7c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.145833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 409/817] 10.222.0.42 () {56 vars in 1115 bytes} [Mon Oct 24 13:01:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:01.152116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.175892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.176097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:01.176497 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.176781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.177060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.177775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:01.178068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.179514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.188083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 409/818] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:01:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:01.193745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.209763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:01.251707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b8fcf-9c6c-4534-9643-a41d6f762499 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.286313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.287034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.287339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.287613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.295320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2738f16-83b5-4485-a92e-712a6c3d604f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.297584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.299016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:01:01.313877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:01.314962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:01:01.315901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:01:01.315901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:01:01.315901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:01:01.315901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 410/819] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:01:01.320774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.329013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595716a-6b03-4bd6-919c-d951132af055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.329639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595716a-6b03-4bd6-919c-d951132af055 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.329933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595716a-6b03-4bd6-919c-d951132af055 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.330207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595716a-6b03-4bd6-919c-d951132af055 None admin] PATH_INFO: `/v3/users/7373d33280fe454098e4d9eeba22d548` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.343560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595716a-6b03-4bd6-919c-d951132af055 None admin] RBAC: Authorizing `identity:get_user(user_id=7373d33280fe454098e4d9eeba22d548)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.344989 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595716a-6b03-4bd6-919c-d951132af055 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.346586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 410/820] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:01 2022] GET /identity/v3/users/7373d33280fe454098e4d9eeba22d548 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:01.352127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595716a-6b03-4bd6-919c-d951132af055 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.397558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:01.419815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f8703-5f96-46f2-8849-e4af2a5e1e03 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.422543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:01.452764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595716a-6b03-4bd6-919c-d951132af055 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.455749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.456354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.456634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.456907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.457541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:01.457843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.459233 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.470778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 411/821] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:01.476340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.486958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c328fb-6f08-4fea-bf9b-46957f478a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.487610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c328fb-6f08-4fea-bf9b-46957f478a69 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.487891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c328fb-6f08-4fea-bf9b-46957f478a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.488164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c328fb-6f08-4fea-bf9b-46957f478a69 None admin] PATH_INFO: `/v3/projects/160ce703bfae4356813b17568d2f7e72` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.489268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c328fb-6f08-4fea-bf9b-46957f478a69 None admin] RBAC: Authorizing `identity:get_project(project_id=160ce703bfae4356813b17568d2f7e72)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.490680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c328fb-6f08-4fea-bf9b-46957f478a69 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.492316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 411/822] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 13:01:01 2022] GET /identity/v3/projects/160ce703bfae4356813b17568d2f7e72 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:01.498053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c328fb-6f08-4fea-bf9b-46957f478a69 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.576782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89e3be-0042-4a65-85c3-63e2cdccbbaf None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.598300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c328fb-6f08-4fea-bf9b-46957f478a69 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.610013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.610633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.610913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.611218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.613820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:01.619097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.620732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.631735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff087f6f-522b-4ed0-acbf-40265bac28b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:01:01.632360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:01.633868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 24 13:01:01.634869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:01:01.635467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff087f6f-522b-4ed0-acbf-40265bac28b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.635467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff087f6f-522b-4ed0-acbf-40265bac28b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.635467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff087f6f-522b-4ed0-acbf-40265bac28b2 None admin] PATH_INFO: `/v3/projects/160ce703bfae4356813b17568d2f7e72/users/7373d33280fe454098e4d9eeba22d548/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.635751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 412/823] 10.222.0.42 () {56 vars in 1093 bytes} [Mon Oct 24 13:01:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:01:01.635826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff087f6f-522b-4ed0-acbf-40265bac28b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=160ce703bfae4356813b17568d2f7e72, user_id=7373d33280fe454098e4d9eeba22d548, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.637114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff087f6f-522b-4ed0-acbf-40265bac28b2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.639743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.646663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:01.652054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 412/824] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:01:01 2022] PUT /identity/v3/projects/160ce703bfae4356813b17568d2f7e72/users/7373d33280fe454098e4d9eeba22d548/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:01:01.759534 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5f139-e989-4e09-afbc-423b107c3011 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.795508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.796126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.796535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.796812 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:01.797552 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:01.797859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:01.799332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:01.819396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 413/825] 10.222.0.42 () {56 vars in 1133 bytes} [Mon Oct 24 13:01:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:01.825488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff087f6f-522b-4ed0-acbf-40265bac28b2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:01.924300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff087f6f-522b-4ed0-acbf-40265bac28b2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:01.956289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b31224-fbdc-48ad-bec8-acc6f9e5008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:01.956893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b31224-fbdc-48ad-bec8-acc6f9e5008f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:01.957184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b31224-fbdc-48ad-bec8-acc6f9e5008f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:01.957450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b31224-fbdc-48ad-bec8-acc6f9e5008f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:02.020979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:02.023603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b31224-fbdc-48ad-bec8-acc6f9e5008f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:02.025830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b31224-fbdc-48ad-bec8-acc6f9e5008f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:02.125464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 413/826] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:01:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:01:02.131492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:02.186281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:02.230917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af67e8b-6143-40ff-abec-c94ad839393a None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:02.264365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:02.264997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:02.265280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:02.265556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:02.275979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:02.277417 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:01:02.291719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:02.292670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:01:02.293600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:01:02.293600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:01:02.293600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:01:02.295210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 414/827] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:01:02.298436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b31224-fbdc-48ad-bec8-acc6f9e5008f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:02.392619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:02.395830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b31224-fbdc-48ad-bec8-acc6f9e5008f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:02.429767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:02.430461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:02.430848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:02.431178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:02.431822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:02.432130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:02.433492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:02.444465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 414/828] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:02.450853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:02.551855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a54f1-6f93-4aa8-90e9-497c1575a695 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:02.559179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:02.588963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:02.589586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:02.589863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:02.590132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:02.596621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:02.598044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:01:02.608897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:02.609981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:02.611088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:01:02.611088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:01:02.611088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:01:02.611088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:01:02.611088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:01:02.611088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:01:02.611088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:01:02.611088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:01:02.614785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 415/829] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:01:02.615810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:02.713220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898a87d-9bc9-4e8e-9d45-7f516a5589c9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:02.745463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d7fa6-09e8-49a0-afa9-d1a48550da50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:02.746054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d7fa6-09e8-49a0-afa9-d1a48550da50 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:02.746332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d7fa6-09e8-49a0-afa9-d1a48550da50 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:02.746600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d7fa6-09e8-49a0-afa9-d1a48550da50 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:02.747307 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d7fa6-09e8-49a0-afa9-d1a48550da50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:02.747612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d7fa6-09e8-49a0-afa9-d1a48550da50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:02.749013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d7fa6-09e8-49a0-afa9-d1a48550da50 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:02.761753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 415/830] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:02.767669 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:02.865289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f9890-95cc-4f81-a653-82300165a8e9 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:02.898022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ea57f-6a44-4e63-924a-2b92948ee5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:02.898619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ea57f-6a44-4e63-924a-2b92948ee5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:02.899469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ea57f-6a44-4e63-924a-2b92948ee5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:02.899739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ea57f-6a44-4e63-924a-2b92948ee5b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:02.900532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ea57f-6a44-4e63-924a-2b92948ee5b8 None admin] RBAC: Adding query filter params (role.id=7a6c27e9bb1d4ab0ae847254d4c8d5de, scope.project.id=00530732ebe84dababafb9f399452940, user.id=2dff78b5eadc42608afbf2b14dd07e7a) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:02.900875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ea57f-6a44-4e63-924a-2b92948ee5b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:02.902377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ea57f-6a44-4e63-924a-2b92948ee5b8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:02.908131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 416/831] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:01:02 2022] GET /identity/v3/role_assignments?user.id=2dff78b5eadc42608afbf2b14dd07e7a&scope.project.id=00530732ebe84dababafb9f399452940&role.id=7a6c27e9bb1d4ab0ae847254d4c8d5de => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:03.809851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ac362-2d03-41e3-b51b-605e0854f9c6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:03.810373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ac362-2d03-41e3-b51b-605e0854f9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:03.810373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ac362-2d03-41e3-b51b-605e0854f9c6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:03.811736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 416/832] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:03.816812 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf5736-a751-4ee7-9238-94ac421620dc None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:03.817238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf5736-a751-4ee7-9238-94ac421620dc None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:03.817366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bf5736-a751-4ee7-9238-94ac421620dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:03.835109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f4bf5736-a751-4ee7-9238-94ac421620dc None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:03.836295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f4bf5736-a751-4ee7-9238-94ac421620dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:03.859429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:03.902813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 417/833] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:04.169507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ac362-2d03-41e3-b51b-605e0854f9c6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:04.191980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:04.270805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ac362-2d03-41e3-b51b-605e0854f9c6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:04.303251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67a18-5d65-4f82-8156-b2ff57f14f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:04.303862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67a18-5d65-4f82-8156-b2ff57f14f28 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:04.304140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67a18-5d65-4f82-8156-b2ff57f14f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:04.304411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67a18-5d65-4f82-8156-b2ff57f14f28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:04.314978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada67a18-5d65-4f82-8156-b2ff57f14f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ada67a18-5d65-4f82-8156-b2ff57f14f28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:01:04.319722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:04.321301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:01:04.322236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 417/834] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:01:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:01:04.328086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf5736-a751-4ee7-9238-94ac421620dc None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:04.424320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bf5736-a751-4ee7-9238-94ac421620dc None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:04.457990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:04.458584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:04.458864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:04.459165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:04.459821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:04.460123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:04.461954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:04.469831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 418/835] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:04.475660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67a18-5d65-4f82-8156-b2ff57f14f28 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:04.557498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:04.576652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67a18-5d65-4f82-8156-b2ff57f14f28 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:04.610090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d66351-ec0a-4cb5-bbc2-0971a5a4998a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:04.610642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d66351-ec0a-4cb5-bbc2-0971a5a4998a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:04.610917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d66351-ec0a-4cb5-bbc2-0971a5a4998a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:04.614463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d66351-ec0a-4cb5-bbc2-0971a5a4998a None admin] PATH_INFO: `/v3/users/7373d33280fe454098e4d9eeba22d548` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:04.614463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d66351-ec0a-4cb5-bbc2-0971a5a4998a None admin] RBAC: Authorizing `identity:get_user(user_id=7373d33280fe454098e4d9eeba22d548)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:04.614463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d66351-ec0a-4cb5-bbc2-0971a5a4998a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:04.615377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 418/836] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:04 2022] GET /identity/v3/users/7373d33280fe454098e4d9eeba22d548 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:04.621211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:04.646391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:04.721545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0a842-624b-4c86-b3c0-d30e1ab25564 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:04.753760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96964edc-7822-4b71-ad44-c6db0affbce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:04.754351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964edc-7822-4b71-ad44-c6db0affbce0 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:04.754629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964edc-7822-4b71-ad44-c6db0affbce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:04.754896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96964edc-7822-4b71-ad44-c6db0affbce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:04.819406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96964edc-7822-4b71-ad44-c6db0affbce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:04.821422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96964edc-7822-4b71-ad44-c6db0affbce0 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:04.844157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:04.918838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 419/837] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:01:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:01:04.924768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d66351-ec0a-4cb5-bbc2-0971a5a4998a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:05.021015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d66351-ec0a-4cb5-bbc2-0971a5a4998a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:05.053105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269aab1a-e23b-4ce9-a66f-20263e48094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:05.053711 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aab1a-e23b-4ce9-a66f-20263e48094d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:05.053992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aab1a-e23b-4ce9-a66f-20263e48094d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:05.054269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269aab1a-e23b-4ce9-a66f-20263e48094d None admin] PATH_INFO: `/v3/projects/160ce703bfae4356813b17568d2f7e72` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:05.055987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269aab1a-e23b-4ce9-a66f-20263e48094d None admin] RBAC: Authorizing `identity:get_project(project_id=160ce703bfae4356813b17568d2f7e72)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:05.057391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269aab1a-e23b-4ce9-a66f-20263e48094d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:05.058982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 419/838] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 13:01:04 2022] GET /identity/v3/projects/160ce703bfae4356813b17568d2f7e72 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:05.065151 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96964edc-7822-4b71-ad44-c6db0affbce0 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:05.161248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96964edc-7822-4b71-ad44-c6db0affbce0 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:05.183778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:05.196007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d44125-be51-44c9-bdc4-c381d1d34ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:05.196585 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44125-be51-44c9-bdc4-c381d1d34ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:05.196861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44125-be51-44c9-bdc4-c381d1d34ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:05.197131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44125-be51-44c9-bdc4-c381d1d34ae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:05.197851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44125-be51-44c9-bdc4-c381d1d34ae0 None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=160ce703bfae4356813b17568d2f7e72, user.id=7373d33280fe454098e4d9eeba22d548) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:05.198157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44125-be51-44c9-bdc4-c381d1d34ae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:05.199614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44125-be51-44c9-bdc4-c381d1d34ae0 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:05.205112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 420/839] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:01:05 2022] GET /identity/v3/role_assignments?user.id=7373d33280fe454098e4d9eeba22d548&scope.project.id=160ce703bfae4356813b17568d2f7e72&role.id=11b74890c7484c6088dd78c106de8be4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:07.365325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd3e1-bb5e-4bae-b93f-874208d50264 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:07.365846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd3e1-bb5e-4bae-b93f-874208d50264 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:07.365846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6cd3e1-bb5e-4bae-b93f-874208d50264 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:07.367226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 420/840] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:07.373056 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c718-fa00-4e7b-a9c7-d1ecb648b7e4 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:07.373471 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c718-fa00-4e7b-a9c7-d1ecb648b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:07.373611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2274c718-fa00-4e7b-a9c7-d1ecb648b7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:07.392279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-2274c718-fa00-4e7b-a9c7-d1ecb648b7e4 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:07.393488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2274c718-fa00-4e7b-a9c7-d1ecb648b7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:07.458497 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 421/841] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:07.731028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cd3e1-bb5e-4bae-b93f-874208d50264 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:07.749534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:07.831161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6cd3e1-bb5e-4bae-b93f-874208d50264 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:07.863923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e372d03-c924-457b-b56c-7b2d775493db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:07.864536 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e372d03-c924-457b-b56c-7b2d775493db None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:07.864822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e372d03-c924-457b-b56c-7b2d775493db None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:07.865099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e372d03-c924-457b-b56c-7b2d775493db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:07.866209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e372d03-c924-457b-b56c-7b2d775493db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:07.867701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e372d03-c924-457b-b56c-7b2d775493db None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:07.869260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 421/842] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:01:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:07.874896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274c718-fa00-4e7b-a9c7-d1ecb648b7e4 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:07.973099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2274c718-fa00-4e7b-a9c7-d1ecb648b7e4 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:07.989186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:08.007412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc4542c-39e0-4d78-bbe3-a1dee90ed8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:08.008623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4542c-39e0-4d78-bbe3-a1dee90ed8c1 None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:08.008903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4542c-39e0-4d78-bbe3-a1dee90ed8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:08.009180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc4542c-39e0-4d78-bbe3-a1dee90ed8c1 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:08.009821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4542c-39e0-4d78-bbe3-a1dee90ed8c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:08.011274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc4542c-39e0-4d78-bbe3-a1dee90ed8c1 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:08.032242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 422/843] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:01:07 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:01:10.224637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a9697-3b79-400a-a6fc-a685f88b998d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:10.225157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a9697-3b79-400a-a6fc-a685f88b998d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:10.225157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a9697-3b79-400a-a6fc-a685f88b998d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:10.226496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 422/844] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:10.232186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aad802-3c74-4b00-9edb-ad1ffdfceb8b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:10.232611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aad802-3c74-4b00-9edb-ad1ffdfceb8b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:10.232752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aad802-3c74-4b00-9edb-ad1ffdfceb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:10.251206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-72aad802-3c74-4b00-9edb-ad1ffdfceb8b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:10.252398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-72aad802-3c74-4b00-9edb-ad1ffdfceb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:10.317905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 423/845] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:10.587663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a9697-3b79-400a-a6fc-a685f88b998d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:10.671678 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:10.688593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a9697-3b79-400a-a6fc-a685f88b998d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:10.722717 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e3aa7-2d19-4a0a-aad0-480f3254472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:10.723372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e3aa7-2d19-4a0a-aad0-480f3254472a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:10.723659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e3aa7-2d19-4a0a-aad0-480f3254472a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:10.723934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e3aa7-2d19-4a0a-aad0-480f3254472a None admin] PATH_INFO: `/v3/roles/78ff1dcdc39743048089711fcfc4d3a4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:10.725318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e3aa7-2d19-4a0a-aad0-480f3254472a None admin] RBAC: Authorizing `identity:get_role(role_id=78ff1dcdc39743048089711fcfc4d3a4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:10.726727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e3aa7-2d19-4a0a-aad0-480f3254472a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:10.728158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 423/846] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:10 2022] GET /identity/v3/roles/78ff1dcdc39743048089711fcfc4d3a4 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:10.733956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aad802-3c74-4b00-9edb-ad1ffdfceb8b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:10.831060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aad802-3c74-4b00-9edb-ad1ffdfceb8b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:10.832538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:10.866254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13ed25-660f-4d31-a966-032b7a528e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:10.866817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13ed25-660f-4d31-a966-032b7a528e1f None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:10.867129 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13ed25-660f-4d31-a966-032b7a528e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:10.867511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13ed25-660f-4d31-a966-032b7a528e1f None admin] PATH_INFO: `/v3/users/541e4f0c80b946cf8b8f70f6ffbea15c` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:10.880196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13ed25-660f-4d31-a966-032b7a528e1f None admin] RBAC: Authorizing `identity:get_user(user_id=541e4f0c80b946cf8b8f70f6ffbea15c)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:10.881872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13ed25-660f-4d31-a966-032b7a528e1f None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:10.884024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 424/847] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:10 2022] GET /identity/v3/users/541e4f0c80b946cf8b8f70f6ffbea15c => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:10.889662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e3aa7-2d19-4a0a-aad0-480f3254472a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:10.985514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e3aa7-2d19-4a0a-aad0-480f3254472a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:11.024626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865fd54-39de-4c9e-a878-113592929efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:11.025230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865fd54-39de-4c9e-a878-113592929efa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:11.025510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865fd54-39de-4c9e-a878-113592929efa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:11.025783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865fd54-39de-4c9e-a878-113592929efa None admin] PATH_INFO: `/v3/projects/160ce703bfae4356813b17568d2f7e72` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:11.026863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865fd54-39de-4c9e-a878-113592929efa None admin] RBAC: Authorizing `identity:get_project(project_id=160ce703bfae4356813b17568d2f7e72)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:11.028295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9865fd54-39de-4c9e-a878-113592929efa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:11.029874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 424/848] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 13:01:10 2022] GET /identity/v3/projects/160ce703bfae4356813b17568d2f7e72 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:11.035623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13ed25-660f-4d31-a966-032b7a528e1f None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:11.135677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13ed25-660f-4d31-a966-032b7a528e1f None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:11.168222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841892f8-773d-4e86-a666-9a18a86792f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:11.168868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841892f8-773d-4e86-a666-9a18a86792f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:11.169149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841892f8-773d-4e86-a666-9a18a86792f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:11.169421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841892f8-773d-4e86-a666-9a18a86792f4 None admin] PATH_INFO: `/v3/projects/160ce703bfae4356813b17568d2f7e72/users/541e4f0c80b946cf8b8f70f6ffbea15c/roles/78ff1dcdc39743048089711fcfc4d3a4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:11.171531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841892f8-773d-4e86-a666-9a18a86792f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=160ce703bfae4356813b17568d2f7e72, user_id=541e4f0c80b946cf8b8f70f6ffbea15c, role_id=78ff1dcdc39743048089711fcfc4d3a4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:11.172953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841892f8-773d-4e86-a666-9a18a86792f4 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:11.185884 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 425/849] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:01:11 2022] PUT /identity/v3/projects/160ce703bfae4356813b17568d2f7e72/users/541e4f0c80b946cf8b8f70f6ffbea15c/roles/78ff1dcdc39743048089711fcfc4d3a4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:01:13.320044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38818-2bcb-4253-8524-a053542edec9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:13.320575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38818-2bcb-4253-8524-a053542edec9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:13.321300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38818-2bcb-4253-8524-a053542edec9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:13.322529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 425/850] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:13.327893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac105d9-431d-4a4b-9b2b-e0a717fc58fb None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:13.328313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac105d9-431d-4a4b-9b2b-e0a717fc58fb None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:13.328407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac105d9-431d-4a4b-9b2b-e0a717fc58fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:13.346076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1ac105d9-431d-4a4b-9b2b-e0a717fc58fb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:13.347283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1ac105d9-431d-4a4b-9b2b-e0a717fc58fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:13.361542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:13.414168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 426/851] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:13.685371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb38818-2bcb-4253-8524-a053542edec9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:13.708588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:13.786705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb38818-2bcb-4253-8524-a053542edec9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:13.818915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855a59a-7932-49c1-8a11-3102d8f152d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:13.819563 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855a59a-7932-49c1-8a11-3102d8f152d4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:13.819849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855a59a-7932-49c1-8a11-3102d8f152d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:13.820128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855a59a-7932-49c1-8a11-3102d8f152d4 None admin] PATH_INFO: `/v3/roles/78ff1dcdc39743048089711fcfc4d3a4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:13.822116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855a59a-7932-49c1-8a11-3102d8f152d4 None admin] RBAC: Authorizing `identity:get_role(role_id=78ff1dcdc39743048089711fcfc4d3a4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:13.823517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5855a59a-7932-49c1-8a11-3102d8f152d4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:13.824830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 426/852] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:13 2022] GET /identity/v3/roles/78ff1dcdc39743048089711fcfc4d3a4 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:13.830557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac105d9-431d-4a4b-9b2b-e0a717fc58fb None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:13.927411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac105d9-431d-4a4b-9b2b-e0a717fc58fb None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:13.961094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e808177-ad77-44b7-b605-5ad45d0b41c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:13.961652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808177-ad77-44b7-b605-5ad45d0b41c8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:13.961935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808177-ad77-44b7-b605-5ad45d0b41c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:13.962208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808177-ad77-44b7-b605-5ad45d0b41c8 None admin] PATH_INFO: `/v3/users/541e4f0c80b946cf8b8f70f6ffbea15c` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:13.963338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e808177-ad77-44b7-b605-5ad45d0b41c8 None admin] RBAC: Authorizing `identity:get_user(user_id=541e4f0c80b946cf8b8f70f6ffbea15c)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:13.964762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e808177-ad77-44b7-b605-5ad45d0b41c8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:13.966319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 427/853] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:13 2022] GET /identity/v3/users/541e4f0c80b946cf8b8f70f6ffbea15c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:13.971941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855a59a-7932-49c1-8a11-3102d8f152d4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:14.058018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:14.071006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855a59a-7932-49c1-8a11-3102d8f152d4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:14.103347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186509c-3626-479b-8d32-b532268e476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:14.103937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186509c-3626-479b-8d32-b532268e476f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:14.104221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186509c-3626-479b-8d32-b532268e476f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:14.104494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186509c-3626-479b-8d32-b532268e476f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:14.176285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186509c-3626-479b-8d32-b532268e476f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:14.177719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186509c-3626-479b-8d32-b532268e476f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:14.262557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:14.274981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 427/854] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:01:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:01:14.281064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e808177-ad77-44b7-b605-5ad45d0b41c8 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:14.300314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:14.379580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e808177-ad77-44b7-b605-5ad45d0b41c8 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:14.412057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe08678-3e62-4c62-9e2b-a82488bf0118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:14.412672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe08678-3e62-4c62-9e2b-a82488bf0118 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:14.412950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe08678-3e62-4c62-9e2b-a82488bf0118 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:14.413219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe08678-3e62-4c62-9e2b-a82488bf0118 None admin] PATH_INFO: `/v3/projects/160ce703bfae4356813b17568d2f7e72` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:14.414830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe08678-3e62-4c62-9e2b-a82488bf0118 None admin] RBAC: Authorizing `identity:get_project(project_id=160ce703bfae4356813b17568d2f7e72)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:14.416269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe08678-3e62-4c62-9e2b-a82488bf0118 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:14.417856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 428/855] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 13:01:14 2022] GET /identity/v3/projects/160ce703bfae4356813b17568d2f7e72 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:14.424113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186509c-3626-479b-8d32-b532268e476f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:14.520336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186509c-3626-479b-8d32-b532268e476f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:14.553036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317440d0-1523-4b81-a594-63fd295b40ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:14.553627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317440d0-1523-4b81-a594-63fd295b40ec None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:14.553910 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317440d0-1523-4b81-a594-63fd295b40ec None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:14.554186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317440d0-1523-4b81-a594-63fd295b40ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:14.555596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317440d0-1523-4b81-a594-63fd295b40ec None admin] RBAC: Adding query filter params (role.id=78ff1dcdc39743048089711fcfc4d3a4, scope.project.id=160ce703bfae4356813b17568d2f7e72, user.id=541e4f0c80b946cf8b8f70f6ffbea15c) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:14.555907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317440d0-1523-4b81-a594-63fd295b40ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:14.557331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317440d0-1523-4b81-a594-63fd295b40ec None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:14.562884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 428/856] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:01:14 2022] GET /identity/v3/role_assignments?user.id=541e4f0c80b946cf8b8f70f6ffbea15c&scope.project.id=160ce703bfae4356813b17568d2f7e72&role.id=78ff1dcdc39743048089711fcfc4d3a4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:16.673985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fb4bd-6173-4d49-9b7c-f006ed8e062e None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:16.674520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fb4bd-6173-4d49-9b7c-f006ed8e062e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:16.674520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fb4bd-6173-4d49-9b7c-f006ed8e062e None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:16.675909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 429/857] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:16.682027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b3bc4-4018-4b9e-ba67-99b4e40de05c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:16.682446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b3bc4-4018-4b9e-ba67-99b4e40de05c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:16.682583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b3bc4-4018-4b9e-ba67-99b4e40de05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:16.700502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3e5b3bc4-4018-4b9e-ba67-99b4e40de05c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:16.701688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3e5b3bc4-4018-4b9e-ba67-99b4e40de05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:16.747297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:16.766793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 429/858] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:17.034659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fb4bd-6173-4d49-9b7c-f006ed8e062e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:17.119072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:17.135163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47fb4bd-6173-4d49-9b7c-f006ed8e062e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:17.168710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291778e-ac76-4a2e-afd9-895796409707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:17.169328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291778e-ac76-4a2e-afd9-895796409707 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:17.169608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291778e-ac76-4a2e-afd9-895796409707 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:17.169883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291778e-ac76-4a2e-afd9-895796409707 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:17.171010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291778e-ac76-4a2e-afd9-895796409707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:17.172456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2291778e-ac76-4a2e-afd9-895796409707 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:17.174004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 430/859] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:01:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:17.179848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b3bc4-4018-4b9e-ba67-99b4e40de05c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:17.274953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b3bc4-4018-4b9e-ba67-99b4e40de05c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:17.307378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0567ff-1383-4707-900c-b8882b33063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:17.307911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0567ff-1383-4707-900c-b8882b33063d None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:17.308190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0567ff-1383-4707-900c-b8882b33063d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:17.308464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0567ff-1383-4707-900c-b8882b33063d None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:17.309108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0567ff-1383-4707-900c-b8882b33063d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:17.310528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0567ff-1383-4707-900c-b8882b33063d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:17.336777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 430/860] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:01:17 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:01:19.489244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5bcfa-08c3-4f11-9723-130930420d5b None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:19.489880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5bcfa-08c3-4f11-9723-130930420d5b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:19.489880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5bcfa-08c3-4f11-9723-130930420d5b None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:19.491150 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 431/861] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:19.496474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949efae8-7b66-49c0-b896-2547228b5687 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:19.496877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949efae8-7b66-49c0-b896-2547228b5687 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:19.497023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949efae8-7b66-49c0-b896-2547228b5687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:19.515628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-949efae8-7b66-49c0-b896-2547228b5687 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:19.516794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-949efae8-7b66-49c0-b896-2547228b5687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:19.542994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:19.582269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 431/862] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:19.860145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5bcfa-08c3-4f11-9723-130930420d5b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:19.960014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5bcfa-08c3-4f11-9723-130930420d5b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:19.992740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac07bd-429b-459a-aafd-f687c0448689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:19.993425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac07bd-429b-459a-aafd-f687c0448689 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:19.993707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac07bd-429b-459a-aafd-f687c0448689 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:19.993991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac07bd-429b-459a-aafd-f687c0448689 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:19.995112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac07bd-429b-459a-aafd-f687c0448689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:19.996531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac07bd-429b-459a-aafd-f687c0448689 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:19.998059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 432/863] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:01:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:20.003757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949efae8-7b66-49c0-b896-2547228b5687 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:20.099988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949efae8-7b66-49c0-b896-2547228b5687 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:20.133890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678fa183-4a08-4d48-8c46-2bf260e9a2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:20.134432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fa183-4a08-4d48-8c46-2bf260e9a2c5 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:20.134714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fa183-4a08-4d48-8c46-2bf260e9a2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:20.134990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678fa183-4a08-4d48-8c46-2bf260e9a2c5 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:20.135661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fa183-4a08-4d48-8c46-2bf260e9a2c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:20.137084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678fa183-4a08-4d48-8c46-2bf260e9a2c5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:20.157285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 432/864] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:01:20 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:01:22.322181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7ba9-187e-4f22-91a9-04d8f40a99db None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:22.322731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7ba9-187e-4f22-91a9-04d8f40a99db None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:22.322731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e7ba9-187e-4f22-91a9-04d8f40a99db None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:22.324089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 433/865] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:22.329456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fcbd1-054e-483d-9156-f4730a3df664 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:22.329865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fcbd1-054e-483d-9156-f4730a3df664 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:22.329969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741fcbd1-054e-483d-9156-f4730a3df664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:22.348965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-741fcbd1-054e-483d-9156-f4730a3df664 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:22.350141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-741fcbd1-054e-483d-9156-f4730a3df664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:22.364318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:22.416804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 433/866] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:22.683702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e7ba9-187e-4f22-91a9-04d8f40a99db None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:22.711743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:22.787603 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e7ba9-187e-4f22-91a9-04d8f40a99db None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:22.820338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd4e5d-7459-4c67-91e9-305d54f410ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:22.820962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd4e5d-7459-4c67-91e9-305d54f410ab None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:22.821241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd4e5d-7459-4c67-91e9-305d54f410ab None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:22.821514 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd4e5d-7459-4c67-91e9-305d54f410ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:22.831565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd4e5d-7459-4c67-91e9-305d54f410ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-95fd4e5d-7459-4c67-91e9-305d54f410ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:01:22.836945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:22.838062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:01:22.839295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 434/867] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:01:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:01:22.844067 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fcbd1-054e-483d-9156-f4730a3df664 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:22.939613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741fcbd1-054e-483d-9156-f4730a3df664 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:22.972089 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:22.972683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:22.972965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:22.973238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:22.973891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:22.974194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:22.975581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:22.983399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 434/868] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:22.989125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd4e5d-7459-4c67-91e9-305d54f410ab None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:23.069537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:23.088836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd4e5d-7459-4c67-91e9-305d54f410ab None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:23.121692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15484d0f-9797-4e03-b2b7-d10291e45969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:23.122311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15484d0f-9797-4e03-b2b7-d10291e45969 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:23.122591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15484d0f-9797-4e03-b2b7-d10291e45969 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:23.122863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15484d0f-9797-4e03-b2b7-d10291e45969 None admin] PATH_INFO: `/v3/users/6f8044ae32a9459ab66b8178a023a33e` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:23.136321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15484d0f-9797-4e03-b2b7-d10291e45969 None admin] RBAC: Authorizing `identity:get_user(user_id=6f8044ae32a9459ab66b8178a023a33e)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:23.137744 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15484d0f-9797-4e03-b2b7-d10291e45969 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:23.139346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 435/869] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:22 2022] GET /identity/v3/users/6f8044ae32a9459ab66b8178a023a33e => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:23.145041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:23.159812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:23.244469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd98c11-d73e-4fdd-9b95-8192184c9818 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:23.276793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4046e-9b86-4bb2-a737-09b4aba62c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:23.277417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4046e-9b86-4bb2-a737-09b4aba62c1a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:23.277698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4046e-9b86-4bb2-a737-09b4aba62c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:23.277973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4046e-9b86-4bb2-a737-09b4aba62c1a None admin] PATH_INFO: `/v3/projects/44e525682e304bd0a6ea977139fe945d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:23.279091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4046e-9b86-4bb2-a737-09b4aba62c1a None admin] RBAC: Authorizing `identity:get_project(project_id=44e525682e304bd0a6ea977139fe945d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:23.280509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4046e-9b86-4bb2-a737-09b4aba62c1a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:23.282111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 435/870] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 13:01:23 2022] GET /identity/v3/projects/44e525682e304bd0a6ea977139fe945d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:23.288208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15484d0f-9797-4e03-b2b7-d10291e45969 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:23.386638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15484d0f-9797-4e03-b2b7-d10291e45969 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:23.420310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbab004-69af-4d4c-96ca-b265adfde798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:23.420951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbab004-69af-4d4c-96ca-b265adfde798 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:23.421231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbab004-69af-4d4c-96ca-b265adfde798 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:23.421505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbab004-69af-4d4c-96ca-b265adfde798 None admin] PATH_INFO: `/v3/projects/44e525682e304bd0a6ea977139fe945d/users/6f8044ae32a9459ab66b8178a023a33e/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:23.423385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbab004-69af-4d4c-96ca-b265adfde798 None admin] RBAC: Authorizing `identity:create_grant(project_id=44e525682e304bd0a6ea977139fe945d, user_id=6f8044ae32a9459ab66b8178a023a33e, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:23.424799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbab004-69af-4d4c-96ca-b265adfde798 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:23.434160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:23.439754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 436/871] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:01:23 2022] PUT /identity/v3/projects/44e525682e304bd0a6ea977139fe945d/users/6f8044ae32a9459ab66b8178a023a33e/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:01:25.595293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fec458-c829-4f37-bd4e-6e158533baa5 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:25.595821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fec458-c829-4f37-bd4e-6e158533baa5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:25.595821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fec458-c829-4f37-bd4e-6e158533baa5 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:25.597174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 436/872] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:25.603361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b255-9df7-48c2-a3be-475fb68e2761 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:25.603783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b255-9df7-48c2-a3be-475fb68e2761 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:25.603918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f2b255-9df7-48c2-a3be-475fb68e2761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:25.622662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-53f2b255-9df7-48c2-a3be-475fb68e2761 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:25.623852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-53f2b255-9df7-48c2-a3be-475fb68e2761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:25.689225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 437/873] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:25.964051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fec458-c829-4f37-bd4e-6e158533baa5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:26.043387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:26.063307 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fec458-c829-4f37-bd4e-6e158533baa5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:26.096745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8ce78-14f9-47ab-9962-c7710d915788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:26.097364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8ce78-14f9-47ab-9962-c7710d915788 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:26.097647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8ce78-14f9-47ab-9962-c7710d915788 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:26.097924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8ce78-14f9-47ab-9962-c7710d915788 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:26.107850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8ce78-14f9-47ab-9962-c7710d915788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b8f8ce78-14f9-47ab-9962-c7710d915788 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:01:26.112619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:26.113669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:01:26.114692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 437/874] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:01:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:01:26.120362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2b255-9df7-48c2-a3be-475fb68e2761 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:26.218134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f2b255-9df7-48c2-a3be-475fb68e2761 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:26.246438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:26.252794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:26.253398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:26.253678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:26.253950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:26.254606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:26.254909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:26.256550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:26.265140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 438/875] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:01:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:26.271147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8ce78-14f9-47ab-9962-c7710d915788 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:26.366261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8ce78-14f9-47ab-9962-c7710d915788 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:26.399392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c4ea2-dabd-4bf5-b511-f50ad60ba4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:26.399988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4ea2-dabd-4bf5-b511-f50ad60ba4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:26.400267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4ea2-dabd-4bf5-b511-f50ad60ba4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:26.400539 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c4ea2-dabd-4bf5-b511-f50ad60ba4e6 None admin] PATH_INFO: `/v3/users/6f8044ae32a9459ab66b8178a023a33e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:26.401630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c4ea2-dabd-4bf5-b511-f50ad60ba4e6 None admin] RBAC: Authorizing `identity:get_user(user_id=6f8044ae32a9459ab66b8178a023a33e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:26.403042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c4ea2-dabd-4bf5-b511-f50ad60ba4e6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:26.404601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 438/876] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:26 2022] GET /identity/v3/users/6f8044ae32a9459ab66b8178a023a33e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:26.410237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:26.509190 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb93d9a-f331-4c1d-99d2-e487965e27dc None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:26.541624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b695d-bbaa-4979-9501-928c3850a7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:26.542210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b695d-bbaa-4979-9501-928c3850a7bc None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:26.542494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b695d-bbaa-4979-9501-928c3850a7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:26.542771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b695d-bbaa-4979-9501-928c3850a7bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:26.598585 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:26.608143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b695d-bbaa-4979-9501-928c3850a7bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:26.609580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b695d-bbaa-4979-9501-928c3850a7bc None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:26.704648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 439/877] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:01:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:01:26.710713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c4ea2-dabd-4bf5-b511-f50ad60ba4e6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:26.722218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:26.809224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c4ea2-dabd-4bf5-b511-f50ad60ba4e6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:26.841790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4db82-2e5e-470a-90c2-fba219bc62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:26.842398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4db82-2e5e-470a-90c2-fba219bc62e6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:26.842677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4db82-2e5e-470a-90c2-fba219bc62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:26.842948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4db82-2e5e-470a-90c2-fba219bc62e6 None admin] PATH_INFO: `/v3/projects/44e525682e304bd0a6ea977139fe945d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:26.844066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4db82-2e5e-470a-90c2-fba219bc62e6 None admin] RBAC: Authorizing `identity:get_project(project_id=44e525682e304bd0a6ea977139fe945d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:26.845481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce4db82-2e5e-470a-90c2-fba219bc62e6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:26.847068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 439/878] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 13:01:26 2022] GET /identity/v3/projects/44e525682e304bd0a6ea977139fe945d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:26.853323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b695d-bbaa-4979-9501-928c3850a7bc None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:26.942326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:26.951825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b695d-bbaa-4979-9501-928c3850a7bc None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:26.984618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bf904-1a29-415a-a642-34f9403dd7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:26.985216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bf904-1a29-415a-a642-34f9403dd7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:26.985499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bf904-1a29-415a-a642-34f9403dd7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:26.985778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bf904-1a29-415a-a642-34f9403dd7e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:26.986499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bf904-1a29-415a-a642-34f9403dd7e5 None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=44e525682e304bd0a6ea977139fe945d, user.id=6f8044ae32a9459ab66b8178a023a33e) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:26.986812 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bf904-1a29-415a-a642-34f9403dd7e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:26.988278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bf904-1a29-415a-a642-34f9403dd7e5 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:26.994447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 440/879] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:01:26 2022] GET /identity/v3/role_assignments?user.id=6f8044ae32a9459ab66b8178a023a33e&scope.project.id=44e525682e304bd0a6ea977139fe945d&role.id=11b74890c7484c6088dd78c106de8be4 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:29.153831 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39c4b-d251-4bcb-a2a5-9927997b41e1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:29.154347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39c4b-d251-4bcb-a2a5-9927997b41e1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:29.154440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39c4b-d251-4bcb-a2a5-9927997b41e1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:29.155727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 440/880] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:29.160816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca29de6-94b8-4553-880e-367ad216f614 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:29.161226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca29de6-94b8-4553-880e-367ad216f614 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:29.161342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca29de6-94b8-4553-880e-367ad216f614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:29.179963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-cca29de6-94b8-4553-880e-367ad216f614 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:29.181140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-cca29de6-94b8-4553-880e-367ad216f614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:29.245801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 441/881] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:29.510754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac39c4b-d251-4bcb-a2a5-9927997b41e1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:29.586732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:29.610171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac39c4b-d251-4bcb-a2a5-9927997b41e1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:29.643177 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37456b-d3f4-43de-9d88-a7622b0e5fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:29.643767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37456b-d3f4-43de-9d88-a7622b0e5fcf None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:29.644054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37456b-d3f4-43de-9d88-a7622b0e5fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:29.644331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37456b-d3f4-43de-9d88-a7622b0e5fcf None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:29.644996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37456b-d3f4-43de-9d88-a7622b0e5fcf None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:29.646355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37456b-d3f4-43de-9d88-a7622b0e5fcf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:29.661699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 441/882] 10.222.0.42 () {60 vars in 1138 bytes} [Mon Oct 24 13:01:29 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:01:31.830143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37c6b-af8f-4c85-8816-1dfd6335ef9a None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:31.830735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37c6b-af8f-4c85-8816-1dfd6335ef9a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:31.830735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad37c6b-af8f-4c85-8816-1dfd6335ef9a None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:31.832092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 442/883] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:31.837969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a008-5887-49d8-a035-091e2a1603e6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:31.838369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a008-5887-49d8-a035-091e2a1603e6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:31.838531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080a008-5887-49d8-a035-091e2a1603e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:31.856211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e080a008-5887-49d8-a035-091e2a1603e6 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:31.857388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e080a008-5887-49d8-a035-091e2a1603e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:31.921387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 442/884] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:32.192070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad37c6b-af8f-4c85-8816-1dfd6335ef9a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:32.276747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:32.293348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad37c6b-af8f-4c85-8816-1dfd6335ef9a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:32.326459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a74f3-b505-404a-b3f9-b38806d2d785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:32.327099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a74f3-b505-404a-b3f9-b38806d2d785 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:32.327389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a74f3-b505-404a-b3f9-b38806d2d785 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:32.327670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a74f3-b505-404a-b3f9-b38806d2d785 None admin] PATH_INFO: `/v3/domains/22c08087c66248019bb180eb56ff8e28` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:32.337975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a74f3-b505-404a-b3f9-b38806d2d785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=22c08087c66248019bb180eb56ff8e28)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:32.339431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a74f3-b505-404a-b3f9-b38806d2d785 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:32.341003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 443/885] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Oct 24 13:01:32 2022] GET /identity/v3/domains/22c08087c66248019bb180eb56ff8e28 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:32.346810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a008-5887-49d8-a035-091e2a1603e6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:32.409935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:32.444648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e080a008-5887-49d8-a035-091e2a1603e6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:32.476767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cd2cb-3a38-4dcc-bb6d-4102f83aa7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:32.477367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd2cb-3a38-4dcc-bb6d-4102f83aa7a2 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:32.477649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd2cb-3a38-4dcc-bb6d-4102f83aa7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:32.477921 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd2cb-3a38-4dcc-bb6d-4102f83aa7a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:32.478557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cd2cb-3a38-4dcc-bb6d-4102f83aa7a2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:32.480025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cd2cb-3a38-4dcc-bb6d-4102f83aa7a2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:32.506411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 443/886] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:01:32 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:01:34.704233 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371ca4-1711-4e26-8182-e9c1c4123d6c None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:34.704784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371ca4-1711-4e26-8182-e9c1c4123d6c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:34.704784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371ca4-1711-4e26-8182-e9c1c4123d6c None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:34.706132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 444/887] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:34.711464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4edc-851d-42ee-82f0-1fd210c10fb9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:34.711857 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4edc-851d-42ee-82f0-1fd210c10fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:34.712011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db4edc-851d-42ee-82f0-1fd210c10fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:34.730126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e7db4edc-851d-42ee-82f0-1fd210c10fb9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:34.731322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e7db4edc-851d-42ee-82f0-1fd210c10fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:34.796291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 444/888] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:35.073688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e371ca4-1711-4e26-8182-e9c1c4123d6c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:35.174547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e371ca4-1711-4e26-8182-e9c1c4123d6c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:35.206551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36e297-59c1-4da2-8733-5392ce340965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:35.207199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36e297-59c1-4da2-8733-5392ce340965 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:35.207483 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36e297-59c1-4da2-8733-5392ce340965 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:35.207759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36e297-59c1-4da2-8733-5392ce340965 None admin] PATH_INFO: `/v3/domains/22c08087c66248019bb180eb56ff8e28` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:35.208858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36e297-59c1-4da2-8733-5392ce340965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=22c08087c66248019bb180eb56ff8e28)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:35.210984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e36e297-59c1-4da2-8733-5392ce340965 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:35.212547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 445/889] 10.222.0.42 () {56 vars in 1183 bytes} [Mon Oct 24 13:01:35 2022] GET /identity/v3/domains/22c08087c66248019bb180eb56ff8e28 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:35.218266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db4edc-851d-42ee-82f0-1fd210c10fb9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:35.266643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:35.317435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db4edc-851d-42ee-82f0-1fd210c10fb9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:35.350423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4fe76a-ab8a-44b5-ab16-d396d0d89a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:35.351588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe76a-ab8a-44b5-ab16-d396d0d89a42 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:35.351955 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe76a-ab8a-44b5-ab16-d396d0d89a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:35.352237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4fe76a-ab8a-44b5-ab16-d396d0d89a42 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:35.352896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fe76a-ab8a-44b5-ab16-d396d0d89a42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:35.354313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4fe76a-ab8a-44b5-ab16-d396d0d89a42 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:35.374843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 445/890] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:01:35 2022] POST /identity/v3/users => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:01:37.542785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b88b8-7ccc-4732-8dbe-a06f89bac6d8 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:37.543361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b88b8-7ccc-4732-8dbe-a06f89bac6d8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:37.543361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b88b8-7ccc-4732-8dbe-a06f89bac6d8 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:37.544689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 446/891] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:37.549865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d01b7b-7aa6-4675-aefd-dc073e6aaae9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:37.550247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d01b7b-7aa6-4675-aefd-dc073e6aaae9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:37.550418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d01b7b-7aa6-4675-aefd-dc073e6aaae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:37.568667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-90d01b7b-7aa6-4675-aefd-dc073e6aaae9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:37.569905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-90d01b7b-7aa6-4675-aefd-dc073e6aaae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:37.634117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 446/892] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:37.904563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b88b8-7ccc-4732-8dbe-a06f89bac6d8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:37.921157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:38.007088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b88b8-7ccc-4732-8dbe-a06f89bac6d8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:38.041479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4381-e87c-4509-831a-65e3b52ebab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:38.042100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4381-e87c-4509-831a-65e3b52ebab3 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:38.042386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4381-e87c-4509-831a-65e3b52ebab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:38.042664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb4381-e87c-4509-831a-65e3b52ebab3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:38.052620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb4381-e87c-4509-831a-65e3b52ebab3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-19eb4381-e87c-4509-831a-65e3b52ebab3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:01:38.057568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:38.058732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:01:38.059835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 447/893] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:01:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:01:38.064701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d01b7b-7aa6-4675-aefd-dc073e6aaae9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:38.097715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:38.165289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d01b7b-7aa6-4675-aefd-dc073e6aaae9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:38.197510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:38.198114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:38.198400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:38.198681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:38.199389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:38.199699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:38.201065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:38.208813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 447/894] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:38.215111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4381-e87c-4509-831a-65e3b52ebab3 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:38.293945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:38.318454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb4381-e87c-4509-831a-65e3b52ebab3 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:38.353997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffea96-c2c7-4400-8ba0-da885f3fb80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:38.354614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffea96-c2c7-4400-8ba0-da885f3fb80b None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:38.354900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffea96-c2c7-4400-8ba0-da885f3fb80b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:38.355210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffea96-c2c7-4400-8ba0-da885f3fb80b None admin] PATH_INFO: `/v3/users/7ea04b97cc61488096536488726fbb56` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:38.368032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffea96-c2c7-4400-8ba0-da885f3fb80b None admin] RBAC: Authorizing `identity:get_user(user_id=7ea04b97cc61488096536488726fbb56)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:38.369469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffea96-c2c7-4400-8ba0-da885f3fb80b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:38.371115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 448/895] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:38 2022] GET /identity/v3/users/7ea04b97cc61488096536488726fbb56 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:38.376826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:38.469435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:38.475475 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d06a9-aeb4-41f2-8b72-a782092cc5e3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:38.508853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3159e5-9c69-4abb-ad2f-d641f097541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:38.509470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3159e5-9c69-4abb-ad2f-d641f097541e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:38.509753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3159e5-9c69-4abb-ad2f-d641f097541e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:38.510027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3159e5-9c69-4abb-ad2f-d641f097541e None admin] PATH_INFO: `/v3/projects/80c8f5096f1042508bdc1cb03f174ab8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:38.511164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3159e5-9c69-4abb-ad2f-d641f097541e None admin] RBAC: Authorizing `identity:get_project(project_id=80c8f5096f1042508bdc1cb03f174ab8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:38.512580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3159e5-9c69-4abb-ad2f-d641f097541e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:38.514202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 448/896] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 13:01:38 2022] GET /identity/v3/projects/80c8f5096f1042508bdc1cb03f174ab8 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:38.520044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffea96-c2c7-4400-8ba0-da885f3fb80b None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:38.616414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffea96-c2c7-4400-8ba0-da885f3fb80b None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:38.645050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:38.651143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01621662-ddc3-48d7-a856-e140a7879141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:38.651785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01621662-ddc3-48d7-a856-e140a7879141 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:38.652064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01621662-ddc3-48d7-a856-e140a7879141 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:38.652337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01621662-ddc3-48d7-a856-e140a7879141 None admin] PATH_INFO: `/v3/projects/80c8f5096f1042508bdc1cb03f174ab8/users/7ea04b97cc61488096536488726fbb56/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:38.654198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01621662-ddc3-48d7-a856-e140a7879141 None admin] RBAC: Authorizing `identity:create_grant(project_id=80c8f5096f1042508bdc1cb03f174ab8, user_id=7ea04b97cc61488096536488726fbb56, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:38.655645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01621662-ddc3-48d7-a856-e140a7879141 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:38.667948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 449/897] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:01:38 2022] PUT /identity/v3/projects/80c8f5096f1042508bdc1cb03f174ab8/users/7ea04b97cc61488096536488726fbb56/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:01:40.819997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8992-9bea-4016-b531-dbe6310c0a3d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:40.820555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8992-9bea-4016-b531-dbe6310c0a3d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:40.820555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb8992-9bea-4016-b531-dbe6310c0a3d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:40.821887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 449/898] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:01:40.827329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1e8e6-a6ed-4b6e-8f8d-849546798902 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:40.827751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1e8e6-a6ed-4b6e-8f8d-849546798902 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:40.827876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1e8e6-a6ed-4b6e-8f8d-849546798902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:40.847079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f2f1e8e6-a6ed-4b6e-8f8d-849546798902 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:01:40.848267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f2f1e8e6-a6ed-4b6e-8f8d-849546798902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:01:40.914608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 450/899] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:01:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:01:41.196108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb8992-9bea-4016-b531-dbe6310c0a3d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:41.296494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb8992-9bea-4016-b531-dbe6310c0a3d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:41.329389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a050f4-7f6d-462c-94ac-93e98bac795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:41.329947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a050f4-7f6d-462c-94ac-93e98bac795b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:41.330226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a050f4-7f6d-462c-94ac-93e98bac795b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:41.330500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a050f4-7f6d-462c-94ac-93e98bac795b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:41.340627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a050f4-7f6d-462c-94ac-93e98bac795b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-04a050f4-7f6d-462c-94ac-93e98bac795b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:01:41.345432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:01:41.346511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:01:41.347545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 450/900] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:01:41.353373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1e8e6-a6ed-4b6e-8f8d-849546798902 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:41.451355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1e8e6-a6ed-4b6e-8f8d-849546798902 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:41.472791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:41.485121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:41.485724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:41.486007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:41.486283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:41.486939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:41.487272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:41.488638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:41.496987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 451/901] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:41.503098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a050f4-7f6d-462c-94ac-93e98bac795b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:41.527968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:41.601545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a050f4-7f6d-462c-94ac-93e98bac795b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:41.635392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370cfcc-f484-4940-b0d4-406bf1021052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:41.636009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370cfcc-f484-4940-b0d4-406bf1021052 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:41.636293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370cfcc-f484-4940-b0d4-406bf1021052 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:41.636570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370cfcc-f484-4940-b0d4-406bf1021052 None admin] PATH_INFO: `/v3/users/7ea04b97cc61488096536488726fbb56` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:41.637676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370cfcc-f484-4940-b0d4-406bf1021052 None admin] RBAC: Authorizing `identity:get_user(user_id=7ea04b97cc61488096536488726fbb56)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:41.639119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370cfcc-f484-4940-b0d4-406bf1021052 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:41.640674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 451/902] 10.222.0.42 () {56 vars in 1177 bytes} [Mon Oct 24 13:01:41 2022] GET /identity/v3/users/7ea04b97cc61488096536488726fbb56 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:41.646389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:41.745029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16a612-0413-4dc6-a3c0-5f83e640dc82 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:41.777821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794479a-d902-41d5-9a8a-2c08469ea3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:41.778408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794479a-d902-41d5-9a8a-2c08469ea3ca None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:41.778694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794479a-d902-41d5-9a8a-2c08469ea3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:41.778973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794479a-d902-41d5-9a8a-2c08469ea3ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:41.828986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:41.845544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794479a-d902-41d5-9a8a-2c08469ea3ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:41.846976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794479a-d902-41d5-9a8a-2c08469ea3ca None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:41.942874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 452/903] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:01:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:01:41.948789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370cfcc-f484-4940-b0d4-406bf1021052 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:42.037427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:42.046263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370cfcc-f484-4940-b0d4-406bf1021052 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:42.080116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994afdc0-64d5-4085-a833-812bd8ac5785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:42.080747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994afdc0-64d5-4085-a833-812bd8ac5785 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:42.081029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994afdc0-64d5-4085-a833-812bd8ac5785 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:42.081306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994afdc0-64d5-4085-a833-812bd8ac5785 None admin] PATH_INFO: `/v3/projects/80c8f5096f1042508bdc1cb03f174ab8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:42.082412 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994afdc0-64d5-4085-a833-812bd8ac5785 None admin] RBAC: Authorizing `identity:get_project(project_id=80c8f5096f1042508bdc1cb03f174ab8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:42.083871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994afdc0-64d5-4085-a833-812bd8ac5785 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:42.086101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 452/904] 10.222.0.42 () {56 vars in 1186 bytes} [Mon Oct 24 13:01:41 2022] GET /identity/v3/projects/80c8f5096f1042508bdc1cb03f174ab8 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:01:42.092264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794479a-d902-41d5-9a8a-2c08469ea3ca None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:01:42.173217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:42.188728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794479a-d902-41d5-9a8a-2c08469ea3ca None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:01:42.222168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1197b57-e292-477a-8b8f-484a4bddac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:01:42.222702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1197b57-e292-477a-8b8f-484a4bddac9d None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:01:42.222983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1197b57-e292-477a-8b8f-484a4bddac9d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:01:42.223292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1197b57-e292-477a-8b8f-484a4bddac9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:01:42.224013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1197b57-e292-477a-8b8f-484a4bddac9d None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=80c8f5096f1042508bdc1cb03f174ab8, user.id=7ea04b97cc61488096536488726fbb56) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:01:42.224320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1197b57-e292-477a-8b8f-484a4bddac9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:01:42.225773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1197b57-e292-477a-8b8f-484a4bddac9d None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:01:42.231441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 453/905] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:01:42 2022] GET /identity/v3/role_assignments?user.id=7ea04b97cc61488096536488726fbb56&scope.project.id=80c8f5096f1042508bdc1cb03f174ab8&role.id=11b74890c7484c6088dd78c106de8be4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:01:52.076987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=20 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:01:52.228887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=26 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:38.308746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eca5e-2c5c-49b2-af55-e2340de000ff None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:38.309383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eca5e-2c5c-49b2-af55-e2340de000ff None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:38.309383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089eca5e-2c5c-49b2-af55-e2340de000ff None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:38.310914 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 453/906] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:02:38.318506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023f92e-ac53-4ba7-b7a3-1a1890026aaa None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:38.319071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023f92e-ac53-4ba7-b7a3-1a1890026aaa None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:38.319288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023f92e-ac53-4ba7-b7a3-1a1890026aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:38.340435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5023f92e-ac53-4ba7-b7a3-1a1890026aaa None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:02:38.341631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5023f92e-ac53-4ba7-b7a3-1a1890026aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:02:38.439112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 454/907] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:02:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:02:38.744088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089eca5e-2c5c-49b2-af55-e2340de000ff None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:38.884721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089eca5e-2c5c-49b2-af55-e2340de000ff None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:38.929567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:38.930216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:38.930557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:38.930908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:38.942817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:38.944483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:38.964132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:38.965220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:38.966201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:38.966201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:38.966201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:38.966201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 454/908] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:38.974099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023f92e-ac53-4ba7-b7a3-1a1890026aaa None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:39.109976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023f92e-ac53-4ba7-b7a3-1a1890026aaa None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:39.144979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:39.152570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace2b65-5797-4483-b974-8b8a2aab3489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:39.153189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2b65-5797-4483-b974-8b8a2aab3489 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:39.153471 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2b65-5797-4483-b974-8b8a2aab3489 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:39.153746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace2b65-5797-4483-b974-8b8a2aab3489 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:39.154402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace2b65-5797-4483-b974-8b8a2aab3489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:39.154707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace2b65-5797-4483-b974-8b8a2aab3489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:39.156114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace2b65-5797-4483-b974-8b8a2aab3489 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:39.172506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 455/909] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:39.178840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:39.271984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:39.334129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffaf144-7408-4571-8941-f35901d98457 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:39.375444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8248677-f097-42ed-a946-6cd7c73e491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:39.376427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248677-f097-42ed-a946-6cd7c73e491a None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:39.376681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248677-f097-42ed-a946-6cd7c73e491a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:39.376962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248677-f097-42ed-a946-6cd7c73e491a None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:39.377842 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8248677-f097-42ed-a946-6cd7c73e491a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:39.379227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8248677-f097-42ed-a946-6cd7c73e491a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:39.404787 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 455/910] 10.222.0.42 () {60 vars in 1132 bytes} [Mon Oct 24 13:02:39 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:02:41.999086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4f09-e433-4e06-9017-4ad888204299 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:41.999638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4f09-e433-4e06-9017-4ad888204299 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:41.999638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4f09-e433-4e06-9017-4ad888204299 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:42.000992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 456/911] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:02:42.008362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd4f78-c308-46e4-bc9b-986c88206c77 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:42.008868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd4f78-c308-46e4-bc9b-986c88206c77 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:42.009118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd4f78-c308-46e4-bc9b-986c88206c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:42.032498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-71bd4f78-c308-46e4-bc9b-986c88206c77 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:02:42.034162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-71bd4f78-c308-46e4-bc9b-986c88206c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:02:42.117837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 456/912] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:02:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:02:42.394590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf4f09-e433-4e06-9017-4ad888204299 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:42.533258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf4f09-e433-4e06-9017-4ad888204299 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:42.580649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c172-df06-476a-8de1-e90aebf4690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:42.581363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c172-df06-476a-8de1-e90aebf4690d None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:42.581650 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c172-df06-476a-8de1-e90aebf4690d None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:42.581927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73c172-df06-476a-8de1-e90aebf4690d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:42.596528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73c172-df06-476a-8de1-e90aebf4690d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-8d73c172-df06-476a-8de1-e90aebf4690d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:02:42.602119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:42.603295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:42.607988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 457/913] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:02:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:42.610010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd4f78-c308-46e4-bc9b-986c88206c77 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:42.659998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:42.739016 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd4f78-c308-46e4-bc9b-986c88206c77 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:42.779680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:42.780289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:42.780621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:42.780907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:42.781783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:42.782091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:42.783677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:42.792806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 457/914] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:02:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:42.798766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c172-df06-476a-8de1-e90aebf4690d None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:42.861648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:42.918525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73c172-df06-476a-8de1-e90aebf4690d None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:42.961836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:42.962471 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:42.962750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:42.963068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:42.975549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:42.976984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:42.992932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:42.994090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:42.995399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:42.995399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:42.995399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:42.995399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 458/915] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:43.000352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:43.110928 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:43.115126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8044c5-a26c-4765-9aad-d9358d93c38f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:43.153465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:43.154160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:43.154464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:43.154943 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:43.155715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:43.156074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:43.157535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:43.174897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 458/916] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:43.180986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:43.303242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37649fee-7ed9-4241-bebb-2be692d6d6bd None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:43.305117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:43.347118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:43.347910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:43.348218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:43.348662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:43.358559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:43.360411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:02:43.374900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:43.376084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:02:43.377046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:43.381673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 459/917] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:43.383937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:43.497947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11b5a1-34ec-4b07-b133-9598c969aae4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:43.543803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:43.544471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:43.544689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:43.544947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:43.545675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:43.545984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:43.547449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:43.566944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 459/918] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Oct 24 13:02:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:43.573037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:43.689231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849b30c-6bfa-4df6-9351-ba516ff6fc38 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:43.734637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:43.735372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:43.735607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:43.735878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:43.748397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:43.749933 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:43.767314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:43.768404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:43.769311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:43.769311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:43.769311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:43.769311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 460/919] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:43.776318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:43.801500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:43.896883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ba5a9-038f-44da-8b85-862f5da3cc6a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:43.931746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79671c-054e-4f34-8415-1d576241274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:43.932353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79671c-054e-4f34-8415-1d576241274e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:43.932623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79671c-054e-4f34-8415-1d576241274e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:43.932897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79671c-054e-4f34-8415-1d576241274e None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:43.933526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79671c-054e-4f34-8415-1d576241274e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:43.933833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79671c-054e-4f34-8415-1d576241274e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:43.935221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79671c-054e-4f34-8415-1d576241274e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:43.947623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 460/920] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:43.953431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:43.970535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:44.073034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6ebe1-d5f2-4a1f-92cd-107e05dfbb0e None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:44.113603 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:44.115420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:44.115905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:44.116691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:44.127329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:44.130765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:02:44.147493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:44.149390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:44.150301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 461/921] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:02:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:02:44.157791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79671c-054e-4f34-8415-1d576241274e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:44.247369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:44.277420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79671c-054e-4f34-8415-1d576241274e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:44.313119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2d1fe-9ec6-428a-ac32-40c60f0b8bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:44.313760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d1fe-9ec6-428a-ac32-40c60f0b8bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:44.314043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d1fe-9ec6-428a-ac32-40c60f0b8bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:44.314321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d1fe-9ec6-428a-ac32-40c60f0b8bf5 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:44.315017 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2d1fe-9ec6-428a-ac32-40c60f0b8bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:44.315368 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2d1fe-9ec6-428a-ac32-40c60f0b8bf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:44.316794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2d1fe-9ec6-428a-ac32-40c60f0b8bf5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:44.329329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 461/922] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:02:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:44.336281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:44.421840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:44.449790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43f143-2097-48fe-a6f1-67b6e537c86c None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:44.487516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaf58f-12ab-4f97-a9ef-91c30b82f5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:44.488328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf58f-12ab-4f97-a9ef-91c30b82f5f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:44.488582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf58f-12ab-4f97-a9ef-91c30b82f5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:44.489001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaf58f-12ab-4f97-a9ef-91c30b82f5f6 None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/81c4bb19721146c7992d8ea11cd165de/roles/6598739f382742d7b956c17b01862204` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:44.505680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf58f-12ab-4f97-a9ef-91c30b82f5f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=81c4bb19721146c7992d8ea11cd165de, role_id=6598739f382742d7b956c17b01862204)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:44.507152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaf58f-12ab-4f97-a9ef-91c30b82f5f6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:44.519279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 462/923] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:02:44 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/81c4bb19721146c7992d8ea11cd165de/roles/6598739f382742d7b956c17b01862204 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:02:46.770086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390e2a-660f-4608-9ff3-d0f81acf160d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:46.770973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390e2a-660f-4608-9ff3-d0f81acf160d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:46.771291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4390e2a-660f-4608-9ff3-d0f81acf160d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:46.772620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 462/924] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:02:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:02:46.777833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f081f4-7fc0-481c-b7ae-5a2f77a3196a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:46.778257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f081f4-7fc0-481c-b7ae-5a2f77a3196a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:46.778376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f081f4-7fc0-481c-b7ae-5a2f77a3196a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:46.797393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-06f081f4-7fc0-481c-b7ae-5a2f77a3196a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:02:46.798595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-06f081f4-7fc0-481c-b7ae-5a2f77a3196a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:02:46.874766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 463/925] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:02:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:02:47.148603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4390e2a-660f-4608-9ff3-d0f81acf160d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:47.261880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4390e2a-660f-4608-9ff3-d0f81acf160d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:47.299390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208135c-6081-4f98-8f99-fb702807e9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:47.300011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208135c-6081-4f98-8f99-fb702807e9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:47.300289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208135c-6081-4f98-8f99-fb702807e9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:47.300560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208135c-6081-4f98-8f99-fb702807e9a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:47.307447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:47.311832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208135c-6081-4f98-8f99-fb702807e9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d208135c-6081-4f98-8f99-fb702807e9a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:02:47.319027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:47.320020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:47.320883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 463/926] 10.222.0.42 () {56 vars in 1102 bytes} [Mon Oct 24 13:02:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:47.327220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f081f4-7fc0-481c-b7ae-5a2f77a3196a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:47.384309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:47.431209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f081f4-7fc0-481c-b7ae-5a2f77a3196a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:47.468938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:47.469657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:47.470025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:47.470423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:47.471377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:47.471747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:47.473622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:47.484177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 464/927] 10.222.0.42 () {56 vars in 1103 bytes} [Mon Oct 24 13:02:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:02:47.491231 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208135c-6081-4f98-8f99-fb702807e9a1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:47.606323 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208135c-6081-4f98-8f99-fb702807e9a1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:47.642947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:47.643553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:47.643868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:47.644181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:47.654646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:47.656269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:47.674103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:47.675258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:47.676346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:47.676346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:47.676346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:47.676346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 464/928] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:02:47.682549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:47.796445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22ca51-1696-4375-ae6c-48cf4ddebcd8 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:47.813756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:47.832210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:47.833799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:47.834082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:47.834361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:47.835004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:47.835342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:47.836718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:47.849933 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 465/929] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:02:47.856167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:47.908287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:47.967354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8acadbb-33c9-4f2f-987d-f23c70e510f5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:48.005519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:48.006223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:48.006524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:48.006827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:48.014855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:48.016422 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:02:48.029655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:48.030725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:48.031700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 465/930] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:48.037637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:48.158664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77818826-fe3c-493a-bcc2-eaf7105d3873 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:48.198789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f479-39de-40b7-8123-515359686a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:48.199498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f479-39de-40b7-8123-515359686a94 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:48.199786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f479-39de-40b7-8123-515359686a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:48.200057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f475f479-39de-40b7-8123-515359686a94 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:48.200857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f479-39de-40b7-8123-515359686a94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:48.201203 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f479-39de-40b7-8123-515359686a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:48.202832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f475f479-39de-40b7-8123-515359686a94 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:48.221955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 466/931] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Oct 24 13:02:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:48.228168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:48.341003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ce331-86e3-472e-a72a-3ea71e15fc63 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:48.353392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:48.380220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90454bdc-c364-4b72-a864-c3d7a6975ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:48.381198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90454bdc-c364-4b72-a864-c3d7a6975ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:48.381503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90454bdc-c364-4b72-a864-c3d7a6975ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:48.381780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90454bdc-c364-4b72-a864-c3d7a6975ad5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:48.453287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90454bdc-c364-4b72-a864-c3d7a6975ad5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:48.454760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90454bdc-c364-4b72-a864-c3d7a6975ad5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:48.565791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 466/932] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:02:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 24 13:02:48.572367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f479-39de-40b7-8123-515359686a94 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:48.611139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:48.687191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f475f479-39de-40b7-8123-515359686a94 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:48.730028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:48.731449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:48.732015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:48.732566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:48.751886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:48.753989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:48.774677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:48.775760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:48.776665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:48.776665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:48.776665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:48.776665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 467/933] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:48.786315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90454bdc-c364-4b72-a864-c3d7a6975ad5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:48.799208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:48.895349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90454bdc-c364-4b72-a864-c3d7a6975ad5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:48.931755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:48.932368 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:48.932650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:48.932925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:48.933605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:48.933977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:48.935646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:48.948597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 467/934] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:02:48.954736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:49.044742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:49.062542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e116e-a33e-43e4-8d12-cced1502a0a4 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:49.107894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:49.108520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:49.108801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:49.109073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:49.116540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:49.117978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:02:49.129974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:49.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:49.132403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 468/935] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:02:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:02:49.138001 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:49.214744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:49.251621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5edc3-81b4-46ac-a06d-8a265578cf87 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:49.289172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a73f3b6-935d-4efb-9014-aede0a85df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:49.289904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f3b6-935d-4efb-9014-aede0a85df63 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:49.290246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f3b6-935d-4efb-9014-aede0a85df63 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:49.290544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a73f3b6-935d-4efb-9014-aede0a85df63 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:49.291415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73f3b6-935d-4efb-9014-aede0a85df63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:49.291730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73f3b6-935d-4efb-9014-aede0a85df63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:49.293328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a73f3b6-935d-4efb-9014-aede0a85df63 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:49.306335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 468/936] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:02:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:49.312396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:49.425074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17b11-1d4b-4ba1-a637-4465eaba4235 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:49.463913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff5934-b00e-40e4-8a82-db38252f46cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:49.465485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff5934-b00e-40e4-8a82-db38252f46cf None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:49.465485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff5934-b00e-40e4-8a82-db38252f46cf None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:49.465485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff5934-b00e-40e4-8a82-db38252f46cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:49.465760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff5934-b00e-40e4-8a82-db38252f46cf None admin] RBAC: Adding query filter params (role.id=6598739f382742d7b956c17b01862204, scope.project.id=00530732ebe84dababafb9f399452940, user.id=81c4bb19721146c7992d8ea11cd165de) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:49.466059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff5934-b00e-40e4-8a82-db38252f46cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:49.467525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff5934-b00e-40e4-8a82-db38252f46cf None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:49.473878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 469/937] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:02:49 2022] GET /identity/v3/role_assignments?user.id=81c4bb19721146c7992d8ea11cd165de&scope.project.id=00530732ebe84dababafb9f399452940&role.id=6598739f382742d7b956c17b01862204 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:51.695529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4203ce0-5072-46f3-bd35-3bce7f4da1d6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:51.696081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4203ce0-5072-46f3-bd35-3bce7f4da1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:51.696081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4203ce0-5072-46f3-bd35-3bce7f4da1d6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:51.697417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 469/938] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:02:51.702802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d12e8-24fa-4634-b72b-2db69830e00a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:51.703366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d12e8-24fa-4634-b72b-2db69830e00a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:51.703366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d12e8-24fa-4634-b72b-2db69830e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:51.724525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-ee7d12e8-24fa-4634-b72b-2db69830e00a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:02:51.725729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ee7d12e8-24fa-4634-b72b-2db69830e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:02:51.728742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:51.802649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 470/939] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:02:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:02:52.080219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4203ce0-5072-46f3-bd35-3bce7f4da1d6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:52.194704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4203ce0-5072-46f3-bd35-3bce7f4da1d6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:52.226278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:52.232621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824c39d-0099-454e-b5e7-fdb40fde4a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:52.233253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824c39d-0099-454e-b5e7-fdb40fde4a1b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:52.233542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824c39d-0099-454e-b5e7-fdb40fde4a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:52.233815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824c39d-0099-454e-b5e7-fdb40fde4a1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:52.245209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824c39d-0099-454e-b5e7-fdb40fde4a1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0824c39d-0099-454e-b5e7-fdb40fde4a1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:02:52.250056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:52.251187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:52.252100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 470/940] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:02:52.257619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d12e8-24fa-4634-b72b-2db69830e00a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:52.376336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d12e8-24fa-4634-b72b-2db69830e00a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:52.410863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:52.411503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:52.411791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:52.412071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:52.412734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:52.413046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:52.414418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:52.422760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 471/941] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:52.429346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824c39d-0099-454e-b5e7-fdb40fde4a1b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:52.553096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824c39d-0099-454e-b5e7-fdb40fde4a1b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:52.592353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:52.593004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:52.593377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:52.593827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:52.605447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:52.606998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:52.622310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:52.623544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:52.624531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:52.624531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:52.624531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:52.624531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 471/942] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:52.631900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:52.635288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:52.752703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f004e-fd0b-494a-8fad-9145a8628894 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:52.793123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:52.793866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:52.794154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:52.794466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:52.795391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:52.795696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:52.797049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:52.810718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 472/943] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:52.818091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:52.868395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:52.934044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415dd09-1c8c-4509-b40b-057a1267ae54 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:52.971082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:52.971745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:52.972023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:52.972307 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:52.981519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:52.982963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:02:52.994629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:53.001313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:53.002675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 472/944] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:53.003374 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:53.076965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:53.122314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab609bb-c55a-49dc-b64d-7a7fc575db91 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:53.158667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:53.159408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:53.159743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:53.160026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:53.161758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:53.162074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:53.163868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:53.182876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 473/945] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Oct 24 13:02:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:53.189003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:53.299722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35561007-f8ef-4713-a5e8-9d9e287c15fb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:53.303045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:53.338760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:53.339516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:53.339795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:53.340073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:53.352443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:53.354058 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:53.371898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:53.372927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:53.373828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:53.373828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:53.373828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:53.373828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 473/946] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:53.379382 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:53.493048 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc8cce-72c6-4426-b462-a4ca468d6dc7 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:53.516476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:53.533548 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:53.534310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:53.534669 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:53.534991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:53.536455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:53.536733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:53.538306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:53.550962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 474/947] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:53.558043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:53.675012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120e31-97e9-45fe-8e86-e2444b28ea7c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:53.715012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:53.715809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:53.716199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:53.716499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:53.725205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:53.727125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:02:53.740832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:53.741942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:53.743325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 474/948] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:02:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:53.748045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:53.864584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885450d-2deb-46d4-a588-48af66a16ad6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:53.904370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd14c3-cdfd-4cfd-9ffe-dd73af1699af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:53.905063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd14c3-cdfd-4cfd-9ffe-dd73af1699af None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:53.905314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd14c3-cdfd-4cfd-9ffe-dd73af1699af None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:53.905609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd14c3-cdfd-4cfd-9ffe-dd73af1699af None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:53.906323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd14c3-cdfd-4cfd-9ffe-dd73af1699af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:53.906631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd14c3-cdfd-4cfd-9ffe-dd73af1699af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:53.908114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd14c3-cdfd-4cfd-9ffe-dd73af1699af None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:53.921410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 475/949] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:02:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:53.928575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:53.946134 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:54.042018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd3fa65-22a3-4450-b91b-6dde6466a48a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:54.084475 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57044431-c536-4086-a50b-5951a71a36c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:54.085179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57044431-c536-4086-a50b-5951a71a36c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:54.085431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57044431-c536-4086-a50b-5951a71a36c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:54.085700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57044431-c536-4086-a50b-5951a71a36c4 None admin] PATH_INFO: `/v3/projects/00530732ebe84dababafb9f399452940/users/81c4bb19721146c7992d8ea11cd165de/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:54.087705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57044431-c536-4086-a50b-5951a71a36c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=00530732ebe84dababafb9f399452940, user_id=81c4bb19721146c7992d8ea11cd165de, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:54.089125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57044431-c536-4086-a50b-5951a71a36c4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:54.101865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 475/950] 10.222.0.42 () {58 vars in 1439 bytes} [Mon Oct 24 13:02:53 2022] PUT /identity/v3/projects/00530732ebe84dababafb9f399452940/users/81c4bb19721146c7992d8ea11cd165de/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:02:56.620750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3db58-42af-4c98-8982-3f17994271b5 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:56.621415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3db58-42af-4c98-8982-3f17994271b5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:56.621415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3db58-42af-4c98-8982-3f17994271b5 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:56.622767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 476/951] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:02:56.629614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5b53c-1ea5-40ae-a3c2-cacf70061c69 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:56.630033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5b53c-1ea5-40ae-a3c2-cacf70061c69 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:56.630480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b5b53c-1ea5-40ae-a3c2-cacf70061c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:56.650827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e8b5b53c-1ea5-40ae-a3c2-cacf70061c69 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:02:56.652487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e8b5b53c-1ea5-40ae-a3c2-cacf70061c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:02:56.739018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 476/952] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:02:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:02:57.040568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3db58-42af-4c98-8982-3f17994271b5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:57.093057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:57.162419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3db58-42af-4c98-8982-3f17994271b5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:57.198490 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acda91b-b8a2-4895-93dc-00d42a6f2375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:57.199154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acda91b-b8a2-4895-93dc-00d42a6f2375 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:57.199437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acda91b-b8a2-4895-93dc-00d42a6f2375 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:57.199709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acda91b-b8a2-4895-93dc-00d42a6f2375 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:57.211101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acda91b-b8a2-4895-93dc-00d42a6f2375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-1acda91b-b8a2-4895-93dc-00d42a6f2375 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 24 13:02:57.216111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:57.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:57.218197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 477/953] 10.222.0.42 () {56 vars in 1096 bytes} [Mon Oct 24 13:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:57.223959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5b53c-1ea5-40ae-a3c2-cacf70061c69 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:57.227175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:57.334758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b5b53c-1ea5-40ae-a3c2-cacf70061c69 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:57.372814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:57.373508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:57.373834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:57.374090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:57.374803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:57.375149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:57.376566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:57.385604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 477/954] 10.222.0.42 () {56 vars in 1099 bytes} [Mon Oct 24 13:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:57.391678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acda91b-b8a2-4895-93dc-00d42a6f2375 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:57.500406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:57.505049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acda91b-b8a2-4895-93dc-00d42a6f2375 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:57.545878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:57.546501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:57.546790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:57.547164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:57.558347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:57.559866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:57.575683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:57.577384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:57.578582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:57.578582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:57.578582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:57.578582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 478/955] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 24 13:02:57.583676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:57.655660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:57.694992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc471220-e239-417b-ac28-91e808ce11a3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:57.741489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:57.742216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:57.742561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:57.744187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:57.744187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:57.744187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:57.745589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:57.761203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 478/956] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:57.767937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:57.883464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d85ea-f49d-4a16-8bc4-2999d13096d8 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:57.919225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c760-fadd-4848-8949-9dca548898af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:57.919791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c760-fadd-4848-8949-9dca548898af None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:57.920072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c760-fadd-4848-8949-9dca548898af None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:57.920346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d323c760-fadd-4848-8949-9dca548898af None admin] PATH_INFO: `/v3/users/placement` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:57.929728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c760-fadd-4848-8949-9dca548898af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:57.931248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d323c760-fadd-4848-8949-9dca548898af None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-d323c760-fadd-4848-8949-9dca548898af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 24 13:02:57.944402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:57.945591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:02:57.946593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 479/957] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:57.951780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:58.057142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0ddc1-e7f5-4e07-a005-ff94ccf0fdf9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:58.089786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:58.100143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:58.100764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:58.101048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:58.101323 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:58.102064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:58.102379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:58.103862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:58.123479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 479/958] 10.222.0.42 () {56 vars in 1143 bytes} [Mon Oct 24 13:02:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:58.129699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c760-fadd-4848-8949-9dca548898af None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:58.146614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:58.235289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d323c760-fadd-4848-8949-9dca548898af None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:58.269192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061b988-f366-42b2-a971-554ffa9d7968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:58.270465 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061b988-f366-42b2-a971-554ffa9d7968 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:58.270751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061b988-f366-42b2-a971-554ffa9d7968 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:58.271027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061b988-f366-42b2-a971-554ffa9d7968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:58.339421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061b988-f366-42b2-a971-554ffa9d7968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:58.340879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c061b988-f366-42b2-a971-554ffa9d7968 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:58.355133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:58.443876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 480/959] 10.222.0.42 () {58 vars in 1282 bytes} [Mon Oct 24 13:02:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 24 13:02:58.449813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:58.546979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567a5ba-cb6e-4998-a884-180b107f68f2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:58.580298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:58.580912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:58.581194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:58.581469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:58.591232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:58.592659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 24 13:02:58.607226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:58.608354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 24 13:02:58.609548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 24 13:02:58.609548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 24 13:02:58.609548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:58.609548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 480/960] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:58.614729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061b988-f366-42b2-a971-554ffa9d7968 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:58.714907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c061b988-f366-42b2-a971-554ffa9d7968 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:58.733794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:58.758157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:58.758768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:58.759047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:58.759354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:58.759994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:58.760295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:58.761647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:58.773701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 481/961] 10.222.0.42 () {56 vars in 1109 bytes} [Mon Oct 24 13:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:02:58.779411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:58.813423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:58.874934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21adac87-ea61-4193-9665-882bb7506330 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:58.907774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:58.908380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:58.908664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:58.908942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:58.915415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:58.916846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 24 13:02:58.928306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:02:58.929372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Oct 24 13:02:58.930648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 481/962] 10.222.0.42 () {56 vars in 1111 bytes} [Mon Oct 24 13:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:02:58.935124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:59.031583 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585b1d10-e935-458c-bccb-649bb0e06d3b None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:59.064792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995c8411-4933-491d-b18a-ba1119111671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:59.065331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c8411-4933-491d-b18a-ba1119111671 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:59.065612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c8411-4933-491d-b18a-ba1119111671 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:59.065884 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c8411-4933-491d-b18a-ba1119111671 None admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:59.066675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c8411-4933-491d-b18a-ba1119111671 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:59.067176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c8411-4933-491d-b18a-ba1119111671 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:59.068377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995c8411-4933-491d-b18a-ba1119111671 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:59.080423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 482/963] 10.222.0.42 () {56 vars in 1148 bytes} [Mon Oct 24 13:02:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:02:59.087003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:02:59.183243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:02:59.189375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172b2a2-fa37-4564-b717-0f3fd1f45b41 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:02:59.221869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e4b66-a450-438a-8f95-1d295ced7a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:02:59.222463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e4b66-a450-438a-8f95-1d295ced7a99 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:02:59.223369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e4b66-a450-438a-8f95-1d295ced7a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:02:59.223737 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e4b66-a450-438a-8f95-1d295ced7a99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:02:59.224462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e4b66-a450-438a-8f95-1d295ced7a99 None admin] RBAC: Adding query filter params (role.id=11b74890c7484c6088dd78c106de8be4, scope.project.id=00530732ebe84dababafb9f399452940, user.id=81c4bb19721146c7992d8ea11cd165de) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:02:59.224773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e4b66-a450-438a-8f95-1d295ced7a99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:02:59.226188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e4b66-a450-438a-8f95-1d295ced7a99 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:02:59.231721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 482/964] 10.222.0.42 () {56 vars in 1374 bytes} [Mon Oct 24 13:02:59 2022] GET /identity/v3/role_assignments?user.id=81c4bb19721146c7992d8ea11cd165de&scope.project.id=00530732ebe84dababafb9f399452940&role.id=11b74890c7484c6088dd78c106de8be4 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:03:01.453648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f403bd9-c841-4deb-91fc-e440d7501491 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:01.454171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f403bd9-c841-4deb-91fc-e440d7501491 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:01.454171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f403bd9-c841-4deb-91fc-e440d7501491 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:01.455579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 483/965] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:03:01.461515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86075644-d905-4c8d-96d8-77b8fa7a6b6b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:01.461912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86075644-d905-4c8d-96d8-77b8fa7a6b6b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:01.462063 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86075644-d905-4c8d-96d8-77b8fa7a6b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:01.480780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-86075644-d905-4c8d-96d8-77b8fa7a6b6b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:03:01.481965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-86075644-d905-4c8d-96d8-77b8fa7a6b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:03:01.546625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 483/966] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:03:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:03:01.823807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f403bd9-c841-4deb-91fc-e440d7501491 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:01.863000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:01.934037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f403bd9-c841-4deb-91fc-e440d7501491 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:01.967022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:01.967667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:01.967946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:01.968218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:01.968790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:01.970144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:03:01.978835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:03:01.980065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:01.981105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 484/967] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 24 13:03:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:03:01.986114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86075644-d905-4c8d-96d8-77b8fa7a6b6b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:02.074141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:02.084120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86075644-d905-4c8d-96d8-77b8fa7a6b6b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:02.117349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65122e20-bf8e-4dd1-9a0d-8a791202d7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:02.117946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65122e20-bf8e-4dd1-9a0d-8a791202d7db None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:02.118236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65122e20-bf8e-4dd1-9a0d-8a791202d7db None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:02.118507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65122e20-bf8e-4dd1-9a0d-8a791202d7db None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:02.119167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65122e20-bf8e-4dd1-9a0d-8a791202d7db None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:03:02.119476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65122e20-bf8e-4dd1-9a0d-8a791202d7db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:02.120834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65122e20-bf8e-4dd1-9a0d-8a791202d7db None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:02.125480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 484/968] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Oct 24 13:03:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:03:02.132527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:02.223743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:02.229828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef01ea1-ec71-4c4b-affa-70f39fd00c03 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:02.265798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26218-f0ed-4a21-842c-84ae2c6d069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:02.266390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26218-f0ed-4a21-842c-84ae2c6d069a None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:02.266668 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26218-f0ed-4a21-842c-84ae2c6d069a None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:02.266937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26218-f0ed-4a21-842c-84ae2c6d069a None admin] PATH_INFO: `/v3/services` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:02.267577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a26218-f0ed-4a21-842c-84ae2c6d069a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:03:02.267877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a26218-f0ed-4a21-842c-84ae2c6d069a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:02.269220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a26218-f0ed-4a21-842c-84ae2c6d069a None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:02.273513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 485/969] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Oct 24 13:03:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:03:04.428733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eb2e2-8981-42c1-aff6-91767f3a35bc None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:04.429266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eb2e2-8981-42c1-aff6-91767f3a35bc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:04.429266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700eb2e2-8981-42c1-aff6-91767f3a35bc None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:04.430620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 485/970] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:03:04.435806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222549c-6974-4bf8-9e68-2e81c40c07f8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:04.436186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222549c-6974-4bf8-9e68-2e81c40c07f8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:04.436360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222549c-6974-4bf8-9e68-2e81c40c07f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:04.454137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f222549c-6974-4bf8-9e68-2e81c40c07f8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:03:04.455334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f222549c-6974-4bf8-9e68-2e81c40c07f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:03:04.521655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 486/971] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:03:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:03:04.786923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eb2e2-8981-42c1-aff6-91767f3a35bc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:04.886314 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700eb2e2-8981-42c1-aff6-91767f3a35bc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:04.920581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478b911-2bba-4628-bc41-eea6fc17cf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:04.921174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478b911-2bba-4628-bc41-eea6fc17cf58 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:04.921457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478b911-2bba-4628-bc41-eea6fc17cf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:04.921759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478b911-2bba-4628-bc41-eea6fc17cf58 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:04.922386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478b911-2bba-4628-bc41-eea6fc17cf58 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:04.923775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478b911-2bba-4628-bc41-eea6fc17cf58 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:04.938213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 486/972] 10.222.0.42 () {60 vars in 1141 bytes} [Mon Oct 24 13:03:04 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:03:07.059488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b59eb-4223-4aa8-8ef9-27326ef290e6 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:07.060046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b59eb-4223-4aa8-8ef9-27326ef290e6 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:07.060046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b59eb-4223-4aa8-8ef9-27326ef290e6 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:07.061373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 487/973] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:03:07.066740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eddc6-beaf-4958-8612-88c09584d170 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:07.067187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eddc6-beaf-4958-8612-88c09584d170 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:07.067319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eddc6-beaf-4958-8612-88c09584d170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:07.085476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-769eddc6-beaf-4958-8612-88c09584d170 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:03:07.086657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-769eddc6-beaf-4958-8612-88c09584d170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:03:07.095933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:07.159264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 487/974] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:03:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:03:07.428004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b59eb-4223-4aa8-8ef9-27326ef290e6 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:07.517333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:07.529520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b59eb-4223-4aa8-8ef9-27326ef290e6 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:07.563253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:07.563875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:07.564156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:07.564433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:07.565008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:07.566507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:03:07.575075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:03:07.576305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:07.577220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 488/975] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 24 13:03:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:03:07.582391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eddc6-beaf-4958-8612-88c09584d170 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:07.679774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eddc6-beaf-4958-8612-88c09584d170 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:07.714662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:07.715321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:07.715622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:07.715862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:07.716531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:03:07.716838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:07.718442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:07.725210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 488/976] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Oct 24 13:03:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:03:07.733330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:07.836167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080043e-c6a7-4d9b-8fe7-dd0a338fa9e6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:07.869793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77036ad1-2c3d-425b-ae72-9e7c4a31e4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:07.870387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036ad1-2c3d-425b-ae72-9e7c4a31e4fb None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:07.870668 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036ad1-2c3d-425b-ae72-9e7c4a31e4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:07.870943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77036ad1-2c3d-425b-ae72-9e7c4a31e4fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:07.871662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77036ad1-2c3d-425b-ae72-9e7c4a31e4fb None admin] RBAC: Adding query filter params (interface=public, service_id=fb30198c831b42029fa1fff9d8481e1f, region_id=RegionOne) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:03:07.871973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77036ad1-2c3d-425b-ae72-9e7c4a31e4fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:07.873323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77036ad1-2c3d-425b-ae72-9e7c4a31e4fb None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:07.878123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 489/977] 10.222.0.42 () {56 vars in 1251 bytes} [Mon Oct 24 13:03:07 2022] GET /identity/v3/endpoints?service_id=fb30198c831b42029fa1fff9d8481e1f&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:03:07.884198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:07.887913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:07.981478 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba736ad-ab47-4189-8e08-efe30882eb1b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:08.014062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fa76b-1234-485b-80ff-4dd795ec68c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:08.014647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa76b-1234-485b-80ff-4dd795ec68c4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:08.014926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa76b-1234-485b-80ff-4dd795ec68c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:08.015233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa76b-1234-485b-80ff-4dd795ec68c4 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:08.015829 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fa76b-1234-485b-80ff-4dd795ec68c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:08.017170 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fa76b-1234-485b-80ff-4dd795ec68c4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:08.024698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 489/978] 10.222.0.42 () {56 vars in 1087 bytes} [Mon Oct 24 13:03:07 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:03:10.184704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba367b-b33b-45aa-b913-6497853093aa None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:10.185243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba367b-b33b-45aa-b913-6497853093aa None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:10.185243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba367b-b33b-45aa-b913-6497853093aa None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:10.186586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 490/979] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:03:10.192553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a12d3-6f2a-45c5-8c7f-b459610a5379 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:10.192969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a12d3-6f2a-45c5-8c7f-b459610a5379 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:10.193082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a12d3-6f2a-45c5-8c7f-b459610a5379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:10.210866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a17a12d3-6f2a-45c5-8c7f-b459610a5379 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:03:10.212087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a17a12d3-6f2a-45c5-8c7f-b459610a5379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:03:10.271888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:10.277715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 490/980] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:03:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 24 13:03:10.549002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba367b-b33b-45aa-b913-6497853093aa None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:10.569247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:10.650430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba367b-b33b-45aa-b913-6497853093aa None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:10.684150 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:10.684771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:10.685060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:10.685338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:10.685917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:10.687302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 24 13:03:10.696000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 24 13:03:10.697245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.get_or_create( Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application with Lock( Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self._enter() Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application return self.creator() Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application created_value = creator( Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: ERROR keystone.server.flask.application Oct 24 13:03:10.698372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 491/981] 10.222.0.42 () {56 vars in 1117 bytes} [Mon Oct 24 13:03:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 24 13:03:10.703715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a12d3-6f2a-45c5-8c7f-b459610a5379 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:10.812658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a12d3-6f2a-45c5-8c7f-b459610a5379 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:10.845609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd1e7b-92ce-4109-a1f5-135fdf738aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:10.846206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1e7b-92ce-4109-a1f5-135fdf738aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:10.846488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1e7b-92ce-4109-a1f5-135fdf738aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:10.846764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd1e7b-92ce-4109-a1f5-135fdf738aa4 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:10.847420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd1e7b-92ce-4109-a1f5-135fdf738aa4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:03:10.847729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd1e7b-92ce-4109-a1f5-135fdf738aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:10.849849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd1e7b-92ce-4109-a1f5-135fdf738aa4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:10.854801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 491/982] 10.222.0.42 () {56 vars in 1116 bytes} [Mon Oct 24 13:03:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:03:10.860626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:03:10.940693 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:10.959716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c7bd-a922-4a7c-8de8-10add609ead6 None admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:03:10.992017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a9e7f-bf2b-43af-bc98-1bdef38263fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:03:10.992606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a9e7f-bf2b-43af-bc98-1bdef38263fc None admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:03:10.992887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a9e7f-bf2b-43af-bc98-1bdef38263fc None admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:03:10.993162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a9e7f-bf2b-43af-bc98-1bdef38263fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:03:10.993814 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a9e7f-bf2b-43af-bc98-1bdef38263fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:03:10.995211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a9e7f-bf2b-43af-bc98-1bdef38263fc None admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:03:11.014222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 492/983] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:03:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:03:20.852367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=69 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:03:21.010128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:15.338522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e01ede-bed9-44db-b096-dad11d80bc3f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:15.338995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e01ede-bed9-44db-b096-dad11d80bc3f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:15.339208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e01ede-bed9-44db-b096-dad11d80bc3f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:15.344451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 492/984] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:15.345846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976012b-30bb-4dcb-a9c0-d0936bf62901 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:15.346266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976012b-30bb-4dcb-a9c0-d0936bf62901 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:15.346574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976012b-30bb-4dcb-a9c0-d0936bf62901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:15.465354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6976012b-30bb-4dcb-a9c0-d0936bf62901 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:15.467157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6976012b-30bb-4dcb-a9c0-d0936bf62901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:15.589476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 493/985] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:04:15.886033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b053f6-63aa-4123-960e-67057645dff7 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:15.886582 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b053f6-63aa-4123-960e-67057645dff7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:15.886970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b053f6-63aa-4123-960e-67057645dff7 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:15.894018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 493/986] 10.222.0.42 () {54 vars in 964 bytes} [Mon Oct 24 13:04:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:15.894972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1caf32-6ba7-48ff-bbf6-9d43c8ba846a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:15.895294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1caf32-6ba7-48ff-bbf6-9d43c8ba846a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:15.895571 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1caf32-6ba7-48ff-bbf6-9d43c8ba846a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:15.983519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0a1caf32-6ba7-48ff-bbf6-9d43c8ba846a None None] MFA Rules not processed for user `9c414b04fe0544e3a938661bc957c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:15.984750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0a1caf32-6ba7-48ff-bbf6-9d43c8ba846a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:16.008768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:16.073372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 494/987] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Oct 24 13:04:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:04:16.082540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b053f6-63aa-4123-960e-67057645dff7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:16.084335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-81b053f6-63aa-4123-960e-67057645dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:16.206869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b053f6-63aa-4123-960e-67057645dff7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:16.290666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3592-a642-4b52-8beb-8cc51e9fbfc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:16.291441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3592-a642-4b52-8beb-8cc51e9fbfc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:16.291729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3592-a642-4b52-8beb-8cc51e9fbfc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:16.292012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3592-a642-4b52-8beb-8cc51e9fbfc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:16.293620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-644e3592-a642-4b52-8beb-8cc51e9fbfc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:16.352611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:16.412971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3592-a642-4b52-8beb-8cc51e9fbfc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:16.414424 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3592-a642-4b52-8beb-8cc51e9fbfc6 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:16.434535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 494/988] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:18.939235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c56ef-a8ff-4fb8-b1b1-bf1fd2613b43 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:18.940494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c56ef-a8ff-4fb8-b1b1-bf1fd2613b43 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:18.940494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c56ef-a8ff-4fb8-b1b1-bf1fd2613b43 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:18.941677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 495/989] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:18.947016 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ced3a-ddc4-443f-b8b0-a39911b56c40 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:18.947460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ced3a-ddc4-443f-b8b0-a39911b56c40 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:18.947608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ced3a-ddc4-443f-b8b0-a39911b56c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:19.028612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f39ced3a-ddc4-443f-b8b0-a39911b56c40 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:19.030213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f39ced3a-ddc4-443f-b8b0-a39911b56c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:19.134261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 495/990] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:19.428440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c56ef-a8ff-4fb8-b1b1-bf1fd2613b43 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:19.429964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-bd6c56ef-a8ff-4fb8-b1b1-bf1fd2613b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:19.534629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c56ef-a8ff-4fb8-b1b1-bf1fd2613b43 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:19.607480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:19.609761 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83babb7b-bb7d-44ad-9ede-f4f683511a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:19.610550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83babb7b-bb7d-44ad-9ede-f4f683511a89 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:19.611007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83babb7b-bb7d-44ad-9ede-f4f683511a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:19.611396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83babb7b-bb7d-44ad-9ede-f4f683511a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:19.613681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-83babb7b-bb7d-44ad-9ede-f4f683511a89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:19.720365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83babb7b-bb7d-44ad-9ede-f4f683511a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:19.722288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83babb7b-bb7d-44ad-9ede-f4f683511a89 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:19.739898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 496/991] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:22.200988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b364-7976-4f0a-afd2-2aa34f358627 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:22.201553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b364-7976-4f0a-afd2-2aa34f358627 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:22.201553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71b364-7976-4f0a-afd2-2aa34f358627 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:22.202856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 496/992] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:22.208130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63e77b-36df-48bf-a514-08254f50ab06 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:22.208536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63e77b-36df-48bf-a514-08254f50ab06 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:22.208670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63e77b-36df-48bf-a514-08254f50ab06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:22.276109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9c63e77b-36df-48bf-a514-08254f50ab06 None None] MFA Rules not processed for user `826adfd9b22449ba95ba52818bac30d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:22.277296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9c63e77b-36df-48bf-a514-08254f50ab06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:22.331150 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:22.376319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 497/993] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:22.662557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71b364-7976-4f0a-afd2-2aa34f358627 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:22.664104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6c71b364-7976-4f0a-afd2-2aa34f358627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:22.674029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:22.767922 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71b364-7976-4f0a-afd2-2aa34f358627 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:22.840285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29670b0a-9924-47cc-ac1f-1705dc13ce4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:22.840882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670b0a-9924-47cc-ac1f-1705dc13ce4e service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:22.841165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670b0a-9924-47cc-ac1f-1705dc13ce4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:22.841438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29670b0a-9924-47cc-ac1f-1705dc13ce4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:22.843022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-29670b0a-9924-47cc-ac1f-1705dc13ce4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:22.905185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:22.952499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29670b0a-9924-47cc-ac1f-1705dc13ce4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:22.953864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29670b0a-9924-47cc-ac1f-1705dc13ce4e service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:22.972720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 497/994] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:25.716792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a091e6d-86ee-4f77-98b4-53a8aabdb726 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:25.717393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a091e6d-86ee-4f77-98b4-53a8aabdb726 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:25.717393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a091e6d-86ee-4f77-98b4-53a8aabdb726 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:25.718671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 498/995] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:25.725963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a5e07-5050-4ffc-b682-30049a00dc1f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:25.726464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a5e07-5050-4ffc-b682-30049a00dc1f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:25.726586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a5e07-5050-4ffc-b682-30049a00dc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:25.792774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-226a5e07-5050-4ffc-b682-30049a00dc1f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:25.793978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-226a5e07-5050-4ffc-b682-30049a00dc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:25.889446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 498/996] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:26.178358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a091e6d-86ee-4f77-98b4-53a8aabdb726 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:26.179939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6a091e6d-86ee-4f77-98b4-53a8aabdb726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:26.283927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a091e6d-86ee-4f77-98b4-53a8aabdb726 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:26.356594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a091d-14b5-49ce-ad69-fbb6d3cb51b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:26.357124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a091d-14b5-49ce-ad69-fbb6d3cb51b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:26.357403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a091d-14b5-49ce-ad69-fbb6d3cb51b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:26.357673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a091d-14b5-49ce-ad69-fbb6d3cb51b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:26.359276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d81a091d-14b5-49ce-ad69-fbb6d3cb51b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:26.366444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:26.468502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a091d-14b5-49ce-ad69-fbb6d3cb51b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:26.469921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a091d-14b5-49ce-ad69-fbb6d3cb51b7 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:26.488793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 499/997] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:29.297889 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f77c0f-349c-48bf-bb10-3403cd21c28e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:29.298555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f77c0f-349c-48bf-bb10-3403cd21c28e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:29.298555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f77c0f-349c-48bf-bb10-3403cd21c28e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:29.300168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 499/998] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:29.305600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63d0d0-e9d9-4093-a20d-5f1de27388f9 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:29.306057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63d0d0-e9d9-4093-a20d-5f1de27388f9 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:29.306143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63d0d0-e9d9-4093-a20d-5f1de27388f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:29.372921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7c63d0d0-e9d9-4093-a20d-5f1de27388f9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:29.374135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c63d0d0-e9d9-4093-a20d-5f1de27388f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:29.442253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:29.484631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 500/999] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:29.788181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ac6f-611b-4f45-82f6-fd4321370089 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:29.788462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ac6f-611b-4f45-82f6-fd4321370089 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:29.788737 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681ac6f-611b-4f45-82f6-fd4321370089 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:29.790090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 500/1000] 10.222.0.42 () {54 vars in 964 bytes} [Mon Oct 24 13:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:29.796251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38076cb-d14c-482d-ad31-0b18f02e5012 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:29.796546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38076cb-d14c-482d-ad31-0b18f02e5012 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:29.796829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38076cb-d14c-482d-ad31-0b18f02e5012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:29.874528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a38076cb-d14c-482d-ad31-0b18f02e5012 None None] MFA Rules not processed for user `9c414b04fe0544e3a938661bc957c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:29.875654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a38076cb-d14c-482d-ad31-0b18f02e5012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:29.951531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 501/1001] 10.222.0.42 () {58 vars in 1061 bytes} [Mon Oct 24 13:04:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:29.961286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681ac6f-611b-4f45-82f6-fd4321370089 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:29.962810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2681ac6f-611b-4f45-82f6-fd4321370089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:29.982767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:30.068726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681ac6f-611b-4f45-82f6-fd4321370089 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:30.149853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4ec5d-5a7c-4fc6-bac3-ed7aea51c3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:30.150362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ec5d-5a7c-4fc6-bac3-ed7aea51c3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:30.150643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ec5d-5a7c-4fc6-bac3-ed7aea51c3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:30.150919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4ec5d-5a7c-4fc6-bac3-ed7aea51c3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:30.152527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fba4ec5d-5a7c-4fc6-bac3-ed7aea51c3a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:30.225982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:30.266136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ec5d-5a7c-4fc6-bac3-ed7aea51c3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:30.267790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4ec5d-5a7c-4fc6-bac3-ed7aea51c3a3 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:30.285706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 501/1002] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:32.747486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490762-f4b1-4420-8f49-f3cbe84dffc2 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:32.748040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490762-f4b1-4420-8f49-f3cbe84dffc2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:32.748160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a490762-f4b1-4420-8f49-f3cbe84dffc2 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:32.749447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 502/1003] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:32.755580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6b90f-8e6b-4afb-9e56-32854f3e9a30 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:32.755981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6b90f-8e6b-4afb-9e56-32854f3e9a30 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:32.756255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6b90f-8e6b-4afb-9e56-32854f3e9a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:32.822229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-11e6b90f-8e6b-4afb-9e56-32854f3e9a30 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:32.823385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-11e6b90f-8e6b-4afb-9e56-32854f3e9a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:32.922360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 502/1004] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:33.203489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a490762-f4b1-4420-8f49-f3cbe84dffc2 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:33.205634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2a490762-f4b1-4420-8f49-f3cbe84dffc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:33.268863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:33.312844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a490762-f4b1-4420-8f49-f3cbe84dffc2 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:33.388675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92381f7e-a624-4628-a6d5-af514c27f1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:33.389206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92381f7e-a624-4628-a6d5-af514c27f1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:33.389496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92381f7e-a624-4628-a6d5-af514c27f1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:33.389791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92381f7e-a624-4628-a6d5-af514c27f1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:33.391416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-92381f7e-a624-4628-a6d5-af514c27f1b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:33.498913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:33.501984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92381f7e-a624-4628-a6d5-af514c27f1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:33.503688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92381f7e-a624-4628-a6d5-af514c27f1b4 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:33.523438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 503/1005] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:35.761234 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735338b-c8be-47f6-8ac4-65654cc394c6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:35.761800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735338b-c8be-47f6-8ac4-65654cc394c6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:35.761800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b735338b-c8be-47f6-8ac4-65654cc394c6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:35.763135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 503/1006] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:35.769120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f60f0-a254-41d9-b5cf-aca5b9dcbf66 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:35.769625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f60f0-a254-41d9-b5cf-aca5b9dcbf66 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:35.769734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934f60f0-a254-41d9-b5cf-aca5b9dcbf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:35.839270 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-934f60f0-a254-41d9-b5cf-aca5b9dcbf66 None None] MFA Rules not processed for user `826adfd9b22449ba95ba52818bac30d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:35.840726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-934f60f0-a254-41d9-b5cf-aca5b9dcbf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:35.945013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 504/1007] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:36.228086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735338b-c8be-47f6-8ac4-65654cc394c6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:36.229616 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b735338b-c8be-47f6-8ac4-65654cc394c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:36.261833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:36.336281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b735338b-c8be-47f6-8ac4-65654cc394c6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:36.414818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4be754-77be-4235-9d58-e5b3e8dd5d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:36.415469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be754-77be-4235-9d58-e5b3e8dd5d65 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:36.415750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be754-77be-4235-9d58-e5b3e8dd5d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:36.416022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be754-77be-4235-9d58-e5b3e8dd5d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:36.417607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5a4be754-77be-4235-9d58-e5b3e8dd5d65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:36.497270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:36.532378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4be754-77be-4235-9d58-e5b3e8dd5d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:36.534726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4be754-77be-4235-9d58-e5b3e8dd5d65 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:36.554492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 504/1008] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:38.959019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02951cf-a8af-4642-a46d-12f9d4eb6f24 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:38.959569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02951cf-a8af-4642-a46d-12f9d4eb6f24 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:38.959670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02951cf-a8af-4642-a46d-12f9d4eb6f24 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:38.960966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 505/1009] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:38.966068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b8015-d9df-4ba1-8e8d-508518603631 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:38.966496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b8015-d9df-4ba1-8e8d-508518603631 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:38.966636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b8015-d9df-4ba1-8e8d-508518603631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:39.031585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-478b8015-d9df-4ba1-8e8d-508518603631 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:39.033144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-478b8015-d9df-4ba1-8e8d-508518603631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:39.131454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 505/1010] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:39.412524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02951cf-a8af-4642-a46d-12f9d4eb6f24 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:39.414046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a02951cf-a8af-4642-a46d-12f9d4eb6f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:39.485197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:39.520480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02951cf-a8af-4642-a46d-12f9d4eb6f24 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:39.596579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692a16a-ae1b-4626-9fdd-de941b467219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:39.597241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692a16a-ae1b-4626-9fdd-de941b467219 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:39.597526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692a16a-ae1b-4626-9fdd-de941b467219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:39.597801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692a16a-ae1b-4626-9fdd-de941b467219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:39.599440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e692a16a-ae1b-4626-9fdd-de941b467219 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:39.709678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692a16a-ae1b-4626-9fdd-de941b467219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:39.711044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692a16a-ae1b-4626-9fdd-de941b467219 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:39.720885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:39.729491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 506/1011] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:41.919315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e962185-5988-4de1-8cb0-7ac91ebe6495 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:41.919846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e962185-5988-4de1-8cb0-7ac91ebe6495 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:41.919846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e962185-5988-4de1-8cb0-7ac91ebe6495 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:41.921209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 506/1012] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:41.927301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1128282-4861-4bcb-8e2e-f4a4b4bf75e0 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:41.927690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1128282-4861-4bcb-8e2e-f4a4b4bf75e0 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:41.927856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1128282-4861-4bcb-8e2e-f4a4b4bf75e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:41.992971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-b1128282-4861-4bcb-8e2e-f4a4b4bf75e0 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:41.994186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b1128282-4861-4bcb-8e2e-f4a4b4bf75e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:42.095999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 507/1013] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:42.386753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e962185-5988-4de1-8cb0-7ac91ebe6495 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:42.388326 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2e962185-5988-4de1-8cb0-7ac91ebe6495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:42.429889 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:42.493442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e962185-5988-4de1-8cb0-7ac91ebe6495 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:42.566254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee2f07-38ad-493a-b2fb-c6c3bd3342e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:42.566861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee2f07-38ad-493a-b2fb-c6c3bd3342e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:42.567176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee2f07-38ad-493a-b2fb-c6c3bd3342e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:42.567457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee2f07-38ad-493a-b2fb-c6c3bd3342e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:42.569055 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-15ee2f07-38ad-493a-b2fb-c6c3bd3342e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:42.651941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:42.678675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee2f07-38ad-493a-b2fb-c6c3bd3342e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:42.680153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee2f07-38ad-493a-b2fb-c6c3bd3342e2 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:42.697723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 507/1014] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:45.217429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a561376-573d-4cc5-8281-613289164d53 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:45.217954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a561376-573d-4cc5-8281-613289164d53 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:45.218068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a561376-573d-4cc5-8281-613289164d53 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:45.219399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 508/1015] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:45.225300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836eca8-fa97-4d15-a2b5-2b2ddd32ede7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:45.225695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836eca8-fa97-4d15-a2b5-2b2ddd32ede7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:45.225869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836eca8-fa97-4d15-a2b5-2b2ddd32ede7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:45.292801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0836eca8-fa97-4d15-a2b5-2b2ddd32ede7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:45.294009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0836eca8-fa97-4d15-a2b5-2b2ddd32ede7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:45.395681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 508/1016] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:45.684430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a561376-573d-4cc5-8281-613289164d53 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:45.686035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2a561376-573d-4cc5-8281-613289164d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:45.765528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:45.793568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a561376-573d-4cc5-8281-613289164d53 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:45.865941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef988fa-7631-438c-92e9-3b44318b4908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:45.866479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef988fa-7631-438c-92e9-3b44318b4908 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:45.866759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef988fa-7631-438c-92e9-3b44318b4908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:45.867032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef988fa-7631-438c-92e9-3b44318b4908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:45.868684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6ef988fa-7631-438c-92e9-3b44318b4908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:45.976310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef988fa-7631-438c-92e9-3b44318b4908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:45.977730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef988fa-7631-438c-92e9-3b44318b4908 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:45.995746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 509/1017] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:50.099278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782939bb-fc59-43d5-ace1-5636457c68a0 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:50.099781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782939bb-fc59-43d5-ace1-5636457c68a0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:50.099781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782939bb-fc59-43d5-ace1-5636457c68a0 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:50.101084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 509/1018] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:50.106389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f941-6eae-4012-ac02-1d820c3b61cd None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:50.106808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f941-6eae-4012-ac02-1d820c3b61cd None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:50.106946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617f941-6eae-4012-ac02-1d820c3b61cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:50.114147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:50.177791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-8617f941-6eae-4012-ac02-1d820c3b61cd None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:50.179001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8617f941-6eae-4012-ac02-1d820c3b61cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:50.277327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 510/1019] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:50.564943 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782939bb-fc59-43d5-ace1-5636457c68a0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:50.566550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-782939bb-fc59-43d5-ace1-5636457c68a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:50.617693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:50.674515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782939bb-fc59-43d5-ace1-5636457c68a0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:50.749095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09804cf-e855-4ccf-ad85-c0f7f23e8f75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:50.749705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09804cf-e855-4ccf-ad85-c0f7f23e8f75 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:50.749986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09804cf-e855-4ccf-ad85-c0f7f23e8f75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:50.750262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09804cf-e855-4ccf-ad85-c0f7f23e8f75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:50.752525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b09804cf-e855-4ccf-ad85-c0f7f23e8f75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:50.846608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:50.864005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09804cf-e855-4ccf-ad85-c0f7f23e8f75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:50.865451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09804cf-e855-4ccf-ad85-c0f7f23e8f75 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:50.886209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 510/1020] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:53.710439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc486d9-303a-413a-9af0-5118f7102801 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:53.711087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc486d9-303a-413a-9af0-5118f7102801 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:53.711087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc486d9-303a-413a-9af0-5118f7102801 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:53.713023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 511/1021] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:04:53.719040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56755ae-73d6-44bb-90df-6e285a94e264 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:53.719502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56755ae-73d6-44bb-90df-6e285a94e264 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:53.719916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56755ae-73d6-44bb-90df-6e285a94e264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:53.793808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d56755ae-73d6-44bb-90df-6e285a94e264 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:53.795017 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d56755ae-73d6-44bb-90df-6e285a94e264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:53.892858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 511/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:04:54.178123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc486d9-303a-413a-9af0-5118f7102801 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:54.179756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-cbc486d9-303a-413a-9af0-5118f7102801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:54.274896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:54.294303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc486d9-303a-413a-9af0-5118f7102801 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:54.370866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e5edf-e0cb-4569-99ac-aa8a6c068fec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:54.373125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e5edf-e0cb-4569-99ac-aa8a6c068fec service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:54.373125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e5edf-e0cb-4569-99ac-aa8a6c068fec service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:54.373125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e5edf-e0cb-4569-99ac-aa8a6c068fec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:54.373633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b28e5edf-e0cb-4569-99ac-aa8a6c068fec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:54.485139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e5edf-e0cb-4569-99ac-aa8a6c068fec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:54.487271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e5edf-e0cb-4569-99ac-aa8a6c068fec service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:54.505935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 512/1023] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:04:58.665285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4defd-f44e-4f6a-89c0-370279189682 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:58.665813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4defd-f44e-4f6a-89c0-370279189682 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:58.665906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc4defd-f44e-4f6a-89c0-370279189682 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:58.667295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 512/1024] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:04:58.673158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0db317-24ac-4ab4-8883-ef8a7bfc7dd8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:58.673740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0db317-24ac-4ab4-8883-ef8a7bfc7dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:58.673740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0db317-24ac-4ab4-8883-ef8a7bfc7dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:58.690111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:58.744741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7f0db317-24ac-4ab4-8883-ef8a7bfc7dd8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:04:58.745942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7f0db317-24ac-4ab4-8883-ef8a7bfc7dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:58.845934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 513/1025] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:04:59.135210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4defd-f44e-4f6a-89c0-370279189682 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:04:59.136741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-afc4defd-f44e-4f6a-89c0-370279189682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:59.198801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:59.245159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc4defd-f44e-4f6a-89c0-370279189682 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:04:59.317369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629e7ff-7a27-45d9-ba60-8b54ca1e05e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:04:59.318033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629e7ff-7a27-45d9-ba60-8b54ca1e05e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:04:59.318261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629e7ff-7a27-45d9-ba60-8b54ca1e05e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:04:59.318537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629e7ff-7a27-45d9-ba60-8b54ca1e05e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:04:59.320159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0629e7ff-7a27-45d9-ba60-8b54ca1e05e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:04:59.431576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:04:59.435820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629e7ff-7a27-45d9-ba60-8b54ca1e05e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:04:59.437278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0629e7ff-7a27-45d9-ba60-8b54ca1e05e4 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:04:59.458467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 513/1026] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:04.521991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc584-859f-4832-98a5-8a511bbe8b49 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:04.522458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc584-859f-4832-98a5-8a511bbe8b49 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:04.522550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc584-859f-4832-98a5-8a511bbe8b49 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:04.524203 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 514/1027] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:04.529341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a2d9-faf9-4128-a806-f663c242ae10 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:04.529727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a2d9-faf9-4128-a806-f663c242ae10 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:04.529907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532a2d9-faf9-4128-a806-f663c242ae10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:04.658463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0532a2d9-faf9-4128-a806-f663c242ae10 None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:04.664972 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0532a2d9-faf9-4128-a806-f663c242ae10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:04.797599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 514/1028] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:05:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:08.843648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=59 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:08.866903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f615cb-773b-484c-84a8-7438b3054e43 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:08.867230 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f615cb-773b-484c-84a8-7438b3054e43 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:08.867549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f615cb-773b-484c-84a8-7438b3054e43 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:08.868897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 515/1029] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:08.874876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313252a-1a30-41dd-b98d-b11b9535b466 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:08.875965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313252a-1a30-41dd-b98d-b11b9535b466 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:08.876187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313252a-1a30-41dd-b98d-b11b9535b466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:08.937074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:08.962251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a313252a-1a30-41dd-b98d-b11b9535b466 None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:08.963499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a313252a-1a30-41dd-b98d-b11b9535b466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:09.082034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 515/1030] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:05:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:10.622069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83100a5b-62da-4ffa-b69f-2b8af2cb0140 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:10.622615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83100a5b-62da-4ffa-b69f-2b8af2cb0140 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:10.622707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83100a5b-62da-4ffa-b69f-2b8af2cb0140 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:10.624189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 516/1031] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:10.630334 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16355e-6615-4424-966e-3f15f7d9c4ba None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:10.630805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16355e-6615-4424-966e-3f15f7d9c4ba None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:10.630901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16355e-6615-4424-966e-3f15f7d9c4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:10.708349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2b16355e-6615-4424-966e-3f15f7d9c4ba None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:10.709620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2b16355e-6615-4424-966e-3f15f7d9c4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:10.754464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:10.820188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 516/1032] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:05:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:14.460722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272bb54-2fda-4e7c-b280-26c020cc3227 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:14.461797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272bb54-2fda-4e7c-b280-26c020cc3227 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:14.462144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272bb54-2fda-4e7c-b280-26c020cc3227 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:14.464732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 517/1033] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:14.473167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096a179-4e76-4e7f-bf73-0f6bcb485343 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:14.473675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096a179-4e76-4e7f-bf73-0f6bcb485343 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:14.473757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096a179-4e76-4e7f-bf73-0f6bcb485343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:14.561521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b096a179-4e76-4e7f-bf73-0f6bcb485343 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:14.562795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b096a179-4e76-4e7f-bf73-0f6bcb485343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:14.672613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 517/1034] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:14.980218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e7b3b-aad9-4f03-b1ce-1ca53629742b None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:14.980688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e7b3b-aad9-4f03-b1ce-1ca53629742b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:14.981144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e7b3b-aad9-4f03-b1ce-1ca53629742b None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:14.982829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 518/1035] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 24 13:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:14.992877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c31a3-7f89-4de3-a348-f18dc61e6df8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:14.993354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c31a3-7f89-4de3-a348-f18dc61e6df8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:14.993997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c31a3-7f89-4de3-a348-f18dc61e6df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:15.025196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:15.099589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3c1c31a3-7f89-4de3-a348-f18dc61e6df8 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:15.101952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3c1c31a3-7f89-4de3-a348-f18dc61e6df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:15.225975 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 518/1036] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 24 13:05:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:05:15.233771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e7b3b-aad9-4f03-b1ce-1ca53629742b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:15.235942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f68e7b3b-aad9-4f03-b1ce-1ca53629742b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:15.393579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e7b3b-aad9-4f03-b1ce-1ca53629742b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:15.471083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:15.516741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a45a4b-6f74-40a2-84b2-c287ac437e9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:15.517445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45a4b-6f74-40a2-84b2-c287ac437e9b service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:15.517799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45a4b-6f74-40a2-84b2-c287ac437e9b service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:15.518140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a45a4b-6f74-40a2-84b2-c287ac437e9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:15.519841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f6a45a4b-6f74-40a2-84b2-c287ac437e9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:15.627289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a45a4b-6f74-40a2-84b2-c287ac437e9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:15.629119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a45a4b-6f74-40a2-84b2-c287ac437e9b service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:15.649368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 519/1037] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:18.069793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c54ff-96bd-4d89-8c3b-29de0e695c4c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:18.069793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c54ff-96bd-4d89-8c3b-29de0e695c4c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:18.070384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c54ff-96bd-4d89-8c3b-29de0e695c4c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:18.076723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca3454-7b71-4092-ae05-ab4d5debc4fe None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:18.076723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca3454-7b71-4092-ae05-ab4d5debc4fe None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:18.077254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca3454-7b71-4092-ae05-ab4d5debc4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:18.082407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 519/1038] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:18 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:18.149016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0cca3454-7b71-4092-ae05-ab4d5debc4fe None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:18.150196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0cca3454-7b71-4092-ae05-ab4d5debc4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:18.191143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:18.258290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 520/1039] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:05:18.553223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c54ff-96bd-4d89-8c3b-29de0e695c4c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:18.554748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-945c54ff-96bd-4d89-8c3b-29de0e695c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:18.630554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:18.740009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c54ff-96bd-4d89-8c3b-29de0e695c4c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:18.887527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fbd5c7-26a1-4d6a-b0e4-57a2516738f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:18.888142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbd5c7-26a1-4d6a-b0e4-57a2516738f7 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:18.888419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbd5c7-26a1-4d6a-b0e4-57a2516738f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:18.888690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fbd5c7-26a1-4d6a-b0e4-57a2516738f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:18.890278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-76fbd5c7-26a1-4d6a-b0e4-57a2516738f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:18.914398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:18.985053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbd5c7-26a1-4d6a-b0e4-57a2516738f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:18.986662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fbd5c7-26a1-4d6a-b0e4-57a2516738f7 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:19.005428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 520/1040] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:21.549203 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205362-9db9-486a-bd3a-053856f9d7b1 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:21.549773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205362-9db9-486a-bd3a-053856f9d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:21.549773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9205362-9db9-486a-bd3a-053856f9d7b1 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:21.553167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 521/1041] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:21.557528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb768b6-a8d8-4495-8b6c-174bef29b7b3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:21.557926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb768b6-a8d8-4495-8b6c-174bef29b7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:21.558095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb768b6-a8d8-4495-8b6c-174bef29b7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:21.628648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1fb768b6-a8d8-4495-8b6c-174bef29b7b3 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:21.630107 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1fb768b6-a8d8-4495-8b6c-174bef29b7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:21.735365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:21.746411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 521/1042] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:22.034706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075aba4-0b68-47c5-9776-1ee43969f405 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:22.034995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075aba4-0b68-47c5-9776-1ee43969f405 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:22.035315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075aba4-0b68-47c5-9776-1ee43969f405 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:22.036644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 522/1043] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 24 13:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:22.042040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f08492-c266-4dff-9fe1-4c9055b2f5ce None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:22.042407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f08492-c266-4dff-9fe1-4c9055b2f5ce None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:22.042906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f08492-c266-4dff-9fe1-4c9055b2f5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:22.127717 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-36f08492-c266-4dff-9fe1-4c9055b2f5ce None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:22.129115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-36f08492-c266-4dff-9fe1-4c9055b2f5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:22.258612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 522/1044] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 24 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:22.266612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075aba4-0b68-47c5-9776-1ee43969f405 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:22.268171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0075aba4-0b68-47c5-9776-1ee43969f405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:22.423997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075aba4-0b68-47c5-9776-1ee43969f405 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:22.445773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:22.553584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5cd5d3-405c-44f7-9c63-9f6ec72c9bd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:22.554244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cd5d3-405c-44f7-9c63-9f6ec72c9bd3 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:22.554469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cd5d3-405c-44f7-9c63-9f6ec72c9bd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:22.554738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5cd5d3-405c-44f7-9c63-9f6ec72c9bd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:22.556618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fb5cd5d3-405c-44f7-9c63-9f6ec72c9bd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:22.663096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5cd5d3-405c-44f7-9c63-9f6ec72c9bd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:22.664731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5cd5d3-405c-44f7-9c63-9f6ec72c9bd3 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:22.685344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 523/1045] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:22.754339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a499-df22-4a9d-a48c-fe44e7c2a090 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:22.755233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a499-df22-4a9d-a48c-fe44e7c2a090 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:22.755233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5a499-df22-4a9d-a48c-fe44e7c2a090 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:22.756835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 523/1046] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:22.762125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb858a7-d89c-425e-a171-e61cd7f0078b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:22.762416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb858a7-d89c-425e-a171-e61cd7f0078b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:22.762694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb858a7-d89c-425e-a171-e61cd7f0078b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:22.804799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:22.851808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-ceb858a7-d89c-425e-a171-e61cd7f0078b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:22.853285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ceb858a7-d89c-425e-a171-e61cd7f0078b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:22.982948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 524/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:05:23.314502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17840308-dd2b-4c47-9196-1000f5493502 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:23.314790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17840308-dd2b-4c47-9196-1000f5493502 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:23.315077 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17840308-dd2b-4c47-9196-1000f5493502 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:23.316896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 524/1048] 10.222.0.42 () {54 vars in 963 bytes} [Mon Oct 24 13:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:23.321914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a46a9-6c87-4353-be35-3d30f7141856 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:23.322208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a46a9-6c87-4353-be35-3d30f7141856 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:23.322485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a46a9-6c87-4353-be35-3d30f7141856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:23.403514 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6b8a46a9-6c87-4353-be35-3d30f7141856 None None] MFA Rules not processed for user `fee3f4d7b69f4851b9160f68a3937703`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:23.404712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6b8a46a9-6c87-4353-be35-3d30f7141856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:23.457378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:23.491091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 525/1049] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Oct 24 13:05:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:05:23.498567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17840308-dd2b-4c47-9196-1000f5493502 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:23.500474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-17840308-dd2b-4c47-9196-1000f5493502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:23.561006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:23.619294 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17840308-dd2b-4c47-9196-1000f5493502 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:23.707002 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4aa70-9bc4-40e6-becf-e1f562a6a50c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:23.707652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4aa70-9bc4-40e6-becf-e1f562a6a50c service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:23.707932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4aa70-9bc4-40e6-becf-e1f562a6a50c service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:23.708213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4aa70-9bc4-40e6-becf-e1f562a6a50c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:23.709847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cea4aa70-9bc4-40e6-becf-e1f562a6a50c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:23.825007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:23.839866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4aa70-9bc4-40e6-becf-e1f562a6a50c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:23.841388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4aa70-9bc4-40e6-becf-e1f562a6a50c service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:23.863665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 525/1050] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:24.952417 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b05d6-b605-4e1e-81c8-fa419c49eba8 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:24.953386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b05d6-b605-4e1e-81c8-fa419c49eba8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:24.953386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b05d6-b605-4e1e-81c8-fa419c49eba8 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:24.954472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 526/1051] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:24.959690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892716f-e4d2-4587-8ac3-ff83e5217b2e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:24.960111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892716f-e4d2-4587-8ac3-ff83e5217b2e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:24.960347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892716f-e4d2-4587-8ac3-ff83e5217b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:25.027423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c892716f-e4d2-4587-8ac3-ff83e5217b2e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:25.028611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c892716f-e4d2-4587-8ac3-ff83e5217b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:25.132967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 526/1052] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:25.424058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b05d6-b605-4e1e-81c8-fa419c49eba8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:25.425638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fe9b05d6-b605-4e1e-81c8-fa419c49eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:25.595465 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b05d6-b605-4e1e-81c8-fa419c49eba8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:25.650875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:25.720371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3da08-092b-4048-afc6-0e2269b39096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:25.720987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3da08-092b-4048-afc6-0e2269b39096 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:25.721268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3da08-092b-4048-afc6-0e2269b39096 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:25.721536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3da08-092b-4048-afc6-0e2269b39096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:25.723189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-24f3da08-092b-4048-afc6-0e2269b39096 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:25.819716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3da08-092b-4048-afc6-0e2269b39096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:25.821408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3da08-092b-4048-afc6-0e2269b39096 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:25.839962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 527/1053] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:05:28.065761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aed36-6943-4ea6-9e76-b05b61d4cb09 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:28.066788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aed36-6943-4ea6-9e76-b05b61d4cb09 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:28.067397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aed36-6943-4ea6-9e76-b05b61d4cb09 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:28.069233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 527/1054] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:28.076322 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166a50a-a96d-4c40-b6aa-4c0748578b5a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:28.076774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166a50a-a96d-4c40-b6aa-4c0748578b5a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:28.076858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166a50a-a96d-4c40-b6aa-4c0748578b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:28.154093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7166a50a-a96d-4c40-b6aa-4c0748578b5a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:28.156345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7166a50a-a96d-4c40-b6aa-4c0748578b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:28.164406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:28.277987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 528/1055] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:28.574138 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aed36-6943-4ea6-9e76-b05b61d4cb09 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:28.575765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c89aed36-6943-4ea6-9e76-b05b61d4cb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:28.642688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:28.737294 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aed36-6943-4ea6-9e76-b05b61d4cb09 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:28.855918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35407f1a-35ad-4902-82d4-0ed7e18b4bac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:28.856441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35407f1a-35ad-4902-82d4-0ed7e18b4bac service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:28.856722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35407f1a-35ad-4902-82d4-0ed7e18b4bac service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:28.856996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35407f1a-35ad-4902-82d4-0ed7e18b4bac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:28.858561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-35407f1a-35ad-4902-82d4-0ed7e18b4bac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:28.878278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:28.955724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35407f1a-35ad-4902-82d4-0ed7e18b4bac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:28.957318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35407f1a-35ad-4902-82d4-0ed7e18b4bac service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:28.976244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 528/1056] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:31.205753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed9502-bf4f-4626-bfa7-77a45223b47c None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:31.206251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed9502-bf4f-4626-bfa7-77a45223b47c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:31.206251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed9502-bf4f-4626-bfa7-77a45223b47c None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:31.207731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 529/1057] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:31.212860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744118a5-9ccc-481a-91b9-575827cb865e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:31.213294 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744118a5-9ccc-481a-91b9-575827cb865e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:31.213642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744118a5-9ccc-481a-91b9-575827cb865e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:31.284993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-744118a5-9ccc-481a-91b9-575827cb865e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:31.286319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-744118a5-9ccc-481a-91b9-575827cb865e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:31.414260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:31.422553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 529/1058] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:31.713128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed9502-bf4f-4626-bfa7-77a45223b47c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:31.714764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-81ed9502-bf4f-4626-bfa7-77a45223b47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:31.860025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:31.871876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed9502-bf4f-4626-bfa7-77a45223b47c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:31.986679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013867a6-e600-4657-b475-a1cf94c2772a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:31.987265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013867a6-e600-4657-b475-a1cf94c2772a service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:31.987549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013867a6-e600-4657-b475-a1cf94c2772a service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:31.987827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013867a6-e600-4657-b475-a1cf94c2772a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:31.989412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-013867a6-e600-4657-b475-a1cf94c2772a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:32.085704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013867a6-e600-4657-b475-a1cf94c2772a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:32.087389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013867a6-e600-4657-b475-a1cf94c2772a service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:32.097860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:32.107072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 530/1059] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:34.789988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7a558-2ccc-4fef-be25-bc0d1aee6a1b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:34.790541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7a558-2ccc-4fef-be25-bc0d1aee6a1b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:34.790541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7a558-2ccc-4fef-be25-bc0d1aee6a1b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:34.791944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 530/1060] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:34.797346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46ea60-832c-4059-8f77-fbfe1e2e5140 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:34.797926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46ea60-832c-4059-8f77-fbfe1e2e5140 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:34.798203 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca46ea60-832c-4059-8f77-fbfe1e2e5140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:34.865442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-ca46ea60-832c-4059-8f77-fbfe1e2e5140 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:34.866506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ca46ea60-832c-4059-8f77-fbfe1e2e5140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:34.965664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 531/1061] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:35.303877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7a558-2ccc-4fef-be25-bc0d1aee6a1b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:35.305401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-74b7a558-2ccc-4fef-be25-bc0d1aee6a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:35.458491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7a558-2ccc-4fef-be25-bc0d1aee6a1b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:35.530009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:35.589768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe2777e-b65d-4724-ac92-829cfc7eae4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:35.590472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2777e-b65d-4724-ac92-829cfc7eae4f service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:35.590843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2777e-b65d-4724-ac92-829cfc7eae4f service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:35.591431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2777e-b65d-4724-ac92-829cfc7eae4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:35.594813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3fe2777e-b65d-4724-ac92-829cfc7eae4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:35.701528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2777e-b65d-4724-ac92-829cfc7eae4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:35.703730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe2777e-b65d-4724-ac92-829cfc7eae4f service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:35.722459 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 531/1062] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:38.235518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af9fd4-705d-49ac-a715-efb18c55aa1f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:38.236107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af9fd4-705d-49ac-a715-efb18c55aa1f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:38.236107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af9fd4-705d-49ac-a715-efb18c55aa1f None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:38.237573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 532/1063] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:38.242642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db837a3d-c330-4c9c-a92d-f3d2f4435f16 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:38.243104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db837a3d-c330-4c9c-a92d-f3d2f4435f16 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:38.243257 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db837a3d-c330-4c9c-a92d-f3d2f4435f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:38.311761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db837a3d-c330-4c9c-a92d-f3d2f4435f16 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:38.313003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db837a3d-c330-4c9c-a92d-f3d2f4435f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:38.347512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:38.413059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 532/1064] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:38.704498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af9fd4-705d-49ac-a715-efb18c55aa1f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:38.706060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a3af9fd4-705d-49ac-a715-efb18c55aa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:38.785977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:38.856865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af9fd4-705d-49ac-a715-efb18c55aa1f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:38.972356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a6986-af12-4577-b1a4-ad74b63f8b55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:38.972898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a6986-af12-4577-b1a4-ad74b63f8b55 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:38.973179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a6986-af12-4577-b1a4-ad74b63f8b55 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:38.973450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a6986-af12-4577-b1a4-ad74b63f8b55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:38.975012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6f6a6986-af12-4577-b1a4-ad74b63f8b55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:39.014124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:39.069953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a6986-af12-4577-b1a4-ad74b63f8b55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:39.071600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a6986-af12-4577-b1a4-ad74b63f8b55 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:39.090978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 533/1065] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:39.712183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8260964-8934-49aa-abb0-a9f1e9cb7651 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:39.712863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8260964-8934-49aa-abb0-a9f1e9cb7651 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:39.712863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8260964-8934-49aa-abb0-a9f1e9cb7651 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:39.714086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 533/1066] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:39.719539 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811c7a-0047-4d73-af9a-c4aa0eea7dda None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:39.720010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811c7a-0047-4d73-af9a-c4aa0eea7dda None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:39.720118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99811c7a-0047-4d73-af9a-c4aa0eea7dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:39.787729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-99811c7a-0047-4d73-af9a-c4aa0eea7dda None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:39.788956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-99811c7a-0047-4d73-af9a-c4aa0eea7dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:39.888469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 534/1067] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:40.201113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aa056-6334-4ffd-aeb1-3fea7413f04c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:40.201401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aa056-6334-4ffd-aeb1-3fea7413f04c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:40.201679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392aa056-6334-4ffd-aeb1-3fea7413f04c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:40.203840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 534/1068] 10.222.0.42 () {54 vars in 962 bytes} [Mon Oct 24 13:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:40.209433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2058-f020-47d6-8f36-23266a2e9e0e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:40.209713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2058-f020-47d6-8f36-23266a2e9e0e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:40.209989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2058-f020-47d6-8f36-23266a2e9e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:40.225642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:40.287515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-ceac2058-f020-47d6-8f36-23266a2e9e0e None None] MFA Rules not processed for user `19d0fc260e254d2d80c6927c52cd8a37`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:40.288727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ceac2058-f020-47d6-8f36-23266a2e9e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:40.364886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 535/1069] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Oct 24 13:05:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:40.374719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aa056-6334-4ffd-aeb1-3fea7413f04c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:40.376465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-392aa056-6334-4ffd-aeb1-3fea7413f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:40.484297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392aa056-6334-4ffd-aeb1-3fea7413f04c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:40.542996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:40.559329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff9c736-121c-4628-89bd-e68bae0f8f58 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d0fc260e254d2d80c6927c52cd8a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:40.559973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9c736-121c-4628-89bd-e68bae0f8f58 service glance] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:40.560255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9c736-121c-4628-89bd-e68bae0f8f58 service glance] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:40.560529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff9c736-121c-4628-89bd-e68bae0f8f58 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:40.562937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3ff9c736-121c-4628-89bd-e68bae0f8f58 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:40.674197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9c736-121c-4628-89bd-e68bae0f8f58 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:40.675708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff9c736-121c-4628-89bd-e68bae0f8f58 service glance] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:40.693766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 535/1070] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:41.311245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db8ddf-52e3-47f3-ae85-ffb59d6f7808 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:41.311913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db8ddf-52e3-47f3-ae85-ffb59d6f7808 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:41.312025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db8ddf-52e3-47f3-ae85-ffb59d6f7808 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:41.313593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 536/1071] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:41.319724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9b59-8349-43f1-9fac-9285ff051ba7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:41.320262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9b59-8349-43f1-9fac-9285ff051ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:41.320262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed9b59-8349-43f1-9fac-9285ff051ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:41.387522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-44ed9b59-8349-43f1-9fac-9285ff051ba7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:41.388741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-44ed9b59-8349-43f1-9fac-9285ff051ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:41.432950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:41.452904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218d4b-d1e5-46c8-b48e-d4dc71e552da None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:41.453288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218d4b-d1e5-46c8-b48e-d4dc71e552da None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:41.453633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2218d4b-d1e5-46c8-b48e-d4dc71e552da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:41.493848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 536/1072] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:41.522955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c2218d4b-d1e5-46c8-b48e-d4dc71e552da None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:41.524190 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c2218d4b-d1e5-46c8-b48e-d4dc71e552da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:41.579115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:41.611352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 537/1073] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:05:41.658565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a5d2a-9d3f-4b2f-8a6d-f8ae6eef182a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:41.658856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a5d2a-9d3f-4b2f-8a6d-f8ae6eef182a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:41.659189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a5d2a-9d3f-4b2f-8a6d-f8ae6eef182a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:41.660814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 537/1074] 10.222.0.42 () {54 vars in 943 bytes} [Mon Oct 24 13:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:41.666559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1726d6-40a0-4e7a-b9d8-0cddab83dd99 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:41.666850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1726d6-40a0-4e7a-b9d8-0cddab83dd99 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:41.667164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1726d6-40a0-4e7a-b9d8-0cddab83dd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:41.747688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4a1726d6-40a0-4e7a-b9d8-0cddab83dd99 None None] MFA Rules not processed for user `5a63131a456548c8a3a3e85dbeba5a48`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:41.748818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4a1726d6-40a0-4e7a-b9d8-0cddab83dd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:41.784269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a5d2a-9d3f-4b2f-8a6d-f8ae6eef182a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:41.785770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b77a5d2a-9d3f-4b2f-8a6d-f8ae6eef182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:41.856587 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 538/1075] 10.222.0.42 () {58 vars in 1040 bytes} [Mon Oct 24 13:05:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:05:41.866213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1726d6-40a0-4e7a-b9d8-0cddab83dd99 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:41.875945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1726d6-40a0-4e7a-b9d8-0cddab83dd99 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:41.904544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:41.935612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a5d2a-9d3f-4b2f-8a6d-f8ae6eef182a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:41.963229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:41.988999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e393c0e-5c14-47e2-82fe-2292104f0e79 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:41.989614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393c0e-5c14-47e2-82fe-2292104f0e79 service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:41.989898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393c0e-5c14-47e2-82fe-2292104f0e79 service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:41.990176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e393c0e-5c14-47e2-82fe-2292104f0e79 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:41.991844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2e393c0e-5c14-47e2-82fe-2292104f0e79 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:42.060134 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d2bb17-0a26-4afd-8ed5-96b00bd49cb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:42.060810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bb17-0a26-4afd-8ed5-96b00bd49cb1 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:42.061224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bb17-0a26-4afd-8ed5-96b00bd49cb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:42.061634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d2bb17-0a26-4afd-8ed5-96b00bd49cb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:42.063748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-72d2bb17-0a26-4afd-8ed5-96b00bd49cb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:42.086839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e393c0e-5c14-47e2-82fe-2292104f0e79 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:42.088485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e393c0e-5c14-47e2-82fe-2292104f0e79 service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:42.097154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 539/1076] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:05:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:05:42.165338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2bb17-0a26-4afd-8ed5-96b00bd49cb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:42.167032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d2bb17-0a26-4afd-8ed5-96b00bd49cb1 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:42.191104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 538/1077] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:05:44.438401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86e9ca-5b5a-4f30-8768-7e467a14adce None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:44.438401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86e9ca-5b5a-4f30-8768-7e467a14adce None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:44.438401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86e9ca-5b5a-4f30-8768-7e467a14adce None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:44.439275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 540/1078] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:44.444100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e2bd-22fc-48b7-89ec-6750e07a1cc4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:44.444528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e2bd-22fc-48b7-89ec-6750e07a1cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:44.444621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e2bd-22fc-48b7-89ec-6750e07a1cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:44.483909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:44.517299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3502e2bd-22fc-48b7-89ec-6750e07a1cc4 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:44.518520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3502e2bd-22fc-48b7-89ec-6750e07a1cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:44.639003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 539/1079] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:44.932444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86e9ca-5b5a-4f30-8768-7e467a14adce None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:44.934037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6b86e9ca-5b5a-4f30-8768-7e467a14adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:45.001352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:45.098473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86e9ca-5b5a-4f30-8768-7e467a14adce None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:45.225495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d60dde-af94-43ae-b2da-3315ee185c6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:45.226098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60dde-af94-43ae-b2da-3315ee185c6d service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:45.226379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60dde-af94-43ae-b2da-3315ee185c6d service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:45.226653 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d60dde-af94-43ae-b2da-3315ee185c6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:45.228463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f1d60dde-af94-43ae-b2da-3315ee185c6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:45.250531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:45.332508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d60dde-af94-43ae-b2da-3315ee185c6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:45.334231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d60dde-af94-43ae-b2da-3315ee185c6d service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:45.353313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 541/1080] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:47.643693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7a281-ea2f-4b40-81a7-692fe145f169 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:47.644226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7a281-ea2f-4b40-81a7-692fe145f169 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:47.644226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7a281-ea2f-4b40-81a7-692fe145f169 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:47.649526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 540/1081] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:47.651026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd77f1-bfb9-435e-b9d4-975bf8af104f None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:47.651492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd77f1-bfb9-435e-b9d4-975bf8af104f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:47.651641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd77f1-bfb9-435e-b9d4-975bf8af104f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:47.728688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d2fd77f1-bfb9-435e-b9d4-975bf8af104f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:47.729912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d2fd77f1-bfb9-435e-b9d4-975bf8af104f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:47.834328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:47.841333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 542/1082] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:05:48.131791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7a281-ea2f-4b40-81a7-692fe145f169 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:48.133377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bfc7a281-ea2f-4b40-81a7-692fe145f169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:48.257753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:48.308477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7a281-ea2f-4b40-81a7-692fe145f169 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:48.440688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bbe52-2274-48f2-be32-c463f597c27c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:48.441302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bbe52-2274-48f2-be32-c463f597c27c service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:48.441648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bbe52-2274-48f2-be32-c463f597c27c service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:48.441930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bbe52-2274-48f2-be32-c463f597c27c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:48.443543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-217bbe52-2274-48f2-be32-c463f597c27c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:48.527666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:48.547866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bbe52-2274-48f2-be32-c463f597c27c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:48.549441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bbe52-2274-48f2-be32-c463f597c27c service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:48.569114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 541/1083] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:05:50.784094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355732d-3bbe-4cd0-9dc4-871593acf01f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:50.784631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355732d-3bbe-4cd0-9dc4-871593acf01f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:50.784631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355732d-3bbe-4cd0-9dc4-871593acf01f None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:50.785990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 543/1084] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:05:50.791203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94cee5-e859-4be1-b712-4fd294c288ba None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:50.791623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94cee5-e859-4be1-b712-4fd294c288ba None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:50.791775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94cee5-e859-4be1-b712-4fd294c288ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:50.865049 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3a94cee5-e859-4be1-b712-4fd294c288ba None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:05:50.866259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3a94cee5-e859-4be1-b712-4fd294c288ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:50.987197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 542/1085] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:05:51.275002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355732d-3bbe-4cd0-9dc4-871593acf01f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:05:51.277029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c355732d-3bbe-4cd0-9dc4-871593acf01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:51.446013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355732d-3bbe-4cd0-9dc4-871593acf01f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:05:51.522838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:05:51.579570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2985897-e3c7-48f6-9d39-0afb6bb6e008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:05:51.580549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985897-e3c7-48f6-9d39-0afb6bb6e008 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:05:51.580843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985897-e3c7-48f6-9d39-0afb6bb6e008 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:05:51.581113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2985897-e3c7-48f6-9d39-0afb6bb6e008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:05:51.582722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e2985897-e3c7-48f6-9d39-0afb6bb6e008 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:05:51.692157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985897-e3c7-48f6-9d39-0afb6bb6e008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:05:51.693678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2985897-e3c7-48f6-9d39-0afb6bb6e008 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:05:51.712947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 544/1086] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:00.979728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=79 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:01.710569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=64 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:04.807357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94023c37-d3d3-4b33-861f-30406bd9f162 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:04.807926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94023c37-d3d3-4b33-861f-30406bd9f162 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:04.807926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94023c37-d3d3-4b33-861f-30406bd9f162 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:04.809240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 543/1087] 10.222.0.42 () {54 vars in 918 bytes} [Mon Oct 24 13:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:04.814820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda52f3-e872-460d-a885-e49c377724aa None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:04.815332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda52f3-e872-460d-a885-e49c377724aa None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:04.815499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda52f3-e872-460d-a885-e49c377724aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:04.891804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-8fda52f3-e872-460d-a885-e49c377724aa None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:04.893047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8fda52f3-e872-460d-a885-e49c377724aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:05.001151 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 545/1088] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Oct 24 13:06:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:05.016185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af477eb-31ca-492e-9202-3ce3e9f5249c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:05.016464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af477eb-31ca-492e-9202-3ce3e9f5249c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:05.016746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af477eb-31ca-492e-9202-3ce3e9f5249c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:05.018066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 544/1089] 10.222.0.42 () {54 vars in 918 bytes} [Mon Oct 24 13:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:05.022681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3220-8137-43c3-8916-4437259a19a2 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:05.023091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3220-8137-43c3-8916-4437259a19a2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:05.023378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3220-8137-43c3-8916-4437259a19a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:05.098036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:05.101067 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d5bd3220-8137-43c3-8916-4437259a19a2 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:05.102964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d5bd3220-8137-43c3-8916-4437259a19a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:05.219957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 546/1090] 10.222.0.42 () {58 vars in 1015 bytes} [Mon Oct 24 13:06:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:06:07.825864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992865c6-63fe-458e-927c-00516e24c12e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:07.826483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992865c6-63fe-458e-927c-00516e24c12e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:07.826483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992865c6-63fe-458e-927c-00516e24c12e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:07.827789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 545/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:07.833125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480af16-7089-42f0-a928-6a79c46a67d7 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:07.833549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480af16-7089-42f0-a928-6a79c46a67d7 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:07.833685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480af16-7089-42f0-a928-6a79c46a67d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:07.906036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c480af16-7089-42f0-a928-6a79c46a67d7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:07.907275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c480af16-7089-42f0-a928-6a79c46a67d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:07.961344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:08.003505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 547/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:08.295455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992865c6-63fe-458e-927c-00516e24c12e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:08.297010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-992865c6-63fe-458e-927c-00516e24c12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:08.441602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992865c6-63fe-458e-927c-00516e24c12e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:08.513229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:08.553454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc12fb-223f-4fa8-9d81-25fd3cd1eaf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:08.554062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc12fb-223f-4fa8-9d81-25fd3cd1eaf4 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:08.554343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc12fb-223f-4fa8-9d81-25fd3cd1eaf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:08.554620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc12fb-223f-4fa8-9d81-25fd3cd1eaf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:08.556243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f0bc12fb-223f-4fa8-9d81-25fd3cd1eaf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:08.644614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc12fb-223f-4fa8-9d81-25fd3cd1eaf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:08.646195 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc12fb-223f-4fa8-9d81-25fd3cd1eaf4 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:08.664793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 546/1093] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:09.804350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bfa17-7524-4483-9a15-83fdde42b2f4 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:09.804899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bfa17-7524-4483-9a15-83fdde42b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:09.804899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074bfa17-7524-4483-9a15-83fdde42b2f4 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:09.806247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 548/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:09.811835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396bac6-a380-4eb1-908b-86a6e663466f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:09.812271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396bac6-a380-4eb1-908b-86a6e663466f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:09.812413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396bac6-a380-4eb1-908b-86a6e663466f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:09.888380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2396bac6-a380-4eb1-908b-86a6e663466f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:09.889704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2396bac6-a380-4eb1-908b-86a6e663466f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:09.916312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:09.986217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 547/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:10.136106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bfa17-7524-4483-9a15-83fdde42b2f4 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:10.137622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-074bfa17-7524-4483-9a15-83fdde42b2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:10.285030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074bfa17-7524-4483-9a15-83fdde42b2f4 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:10.325144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:10.400113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a88b5-e9c9-40bf-806e-970ce0df6d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:10.400719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a88b5-e9c9-40bf-806e-970ce0df6d8a service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:10.400997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a88b5-e9c9-40bf-806e-970ce0df6d8a service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:10.401267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a88b5-e9c9-40bf-806e-970ce0df6d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:10.402853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-bf2a88b5-e9c9-40bf-806e-970ce0df6d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:10.492399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a88b5-e9c9-40bf-806e-970ce0df6d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:10.493974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a88b5-e9c9-40bf-806e-970ce0df6d8a service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:10.511997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 549/1096] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:06:11.631944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83953c-4174-4623-a2e9-69dfa4c0dc94 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:11.632513 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83953c-4174-4623-a2e9-69dfa4c0dc94 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:11.632513 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83953c-4174-4623-a2e9-69dfa4c0dc94 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:11.633824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 548/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:11.639214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dfdf6-f294-4566-95ae-a951042fd3e8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:11.639621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dfdf6-f294-4566-95ae-a951042fd3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:11.639775 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dfdf6-f294-4566-95ae-a951042fd3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:11.694228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:11.717365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f56dfdf6-f294-4566-95ae-a951042fd3e8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:11.718588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f56dfdf6-f294-4566-95ae-a951042fd3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:11.814546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 550/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:11.971036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83953c-4174-4623-a2e9-69dfa4c0dc94 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:11.972595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9e83953c-4174-4623-a2e9-69dfa4c0dc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:12.121141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83953c-4174-4623-a2e9-69dfa4c0dc94 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:12.134256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:12.234958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5000239-6a50-4954-a942-982ae54ac522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:12.235508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5000239-6a50-4954-a942-982ae54ac522 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:12.235794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5000239-6a50-4954-a942-982ae54ac522 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:12.236069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5000239-6a50-4954-a942-982ae54ac522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:12.237664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f5000239-6a50-4954-a942-982ae54ac522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:12.327814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5000239-6a50-4954-a942-982ae54ac522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:12.329411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5000239-6a50-4954-a942-982ae54ac522 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:12.346961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 549/1099] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:13.447805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad7bfd-bc3a-4065-ad94-13b8359ac62f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:13.448385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad7bfd-bc3a-4065-ad94-13b8359ac62f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:13.448385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fad7bfd-bc3a-4065-ad94-13b8359ac62f None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:13.449731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 551/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:13.454690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0253ad-f2ad-4f13-ba1d-1ae6da73f179 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:13.455149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0253ad-f2ad-4f13-ba1d-1ae6da73f179 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:13.455344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0253ad-f2ad-4f13-ba1d-1ae6da73f179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:13.475303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:13.533342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7b0253ad-f2ad-4f13-ba1d-1ae6da73f179 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:13.534548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7b0253ad-f2ad-4f13-ba1d-1ae6da73f179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:13.630262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 550/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:13.781338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad7bfd-bc3a-4065-ad94-13b8359ac62f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:13.782845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3fad7bfd-bc3a-4065-ad94-13b8359ac62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:13.892735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:13.929436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fad7bfd-bc3a-4065-ad94-13b8359ac62f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:14.043615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d1131-8895-42c1-9fbf-82f0f5d332df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:14.044158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d1131-8895-42c1-9fbf-82f0f5d332df service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:14.044466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d1131-8895-42c1-9fbf-82f0f5d332df service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:14.044819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d1131-8895-42c1-9fbf-82f0f5d332df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:14.046428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-891d1131-8895-42c1-9fbf-82f0f5d332df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:14.125745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:14.140538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d1131-8895-42c1-9fbf-82f0f5d332df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:14.142167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d1131-8895-42c1-9fbf-82f0f5d332df service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:14.159912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 552/1102] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:15.266324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277f62-6f64-4196-a2ad-95512fa24fd5 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:15.266941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277f62-6f64-4196-a2ad-95512fa24fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:15.266941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c277f62-6f64-4196-a2ad-95512fa24fd5 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:15.268167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 551/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:15.272989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e272d3c-c647-4fe9-9aa8-27c2c368bacb None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:15.273432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e272d3c-c647-4fe9-9aa8-27c2c368bacb None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:15.273588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e272d3c-c647-4fe9-9aa8-27c2c368bacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:15.348872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5e272d3c-c647-4fe9-9aa8-27c2c368bacb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:15.350079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5e272d3c-c647-4fe9-9aa8-27c2c368bacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:15.446546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 553/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:15.598699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c277f62-6f64-4196-a2ad-95512fa24fd5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:15.600272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6c277f62-6f64-4196-a2ad-95512fa24fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:15.683568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:15.746158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c277f62-6f64-4196-a2ad-95512fa24fd5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:15.858605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cf1ba-72ab-4342-b644-3b4a9f3714c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:15.859164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf1ba-72ab-4342-b644-3b4a9f3714c8 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:15.859454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf1ba-72ab-4342-b644-3b4a9f3714c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:15.859730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cf1ba-72ab-4342-b644-3b4a9f3714c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:15.861324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a73cf1ba-72ab-4342-b644-3b4a9f3714c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:15.909416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:15.950752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf1ba-72ab-4342-b644-3b4a9f3714c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:15.952532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cf1ba-72ab-4342-b644-3b4a9f3714c8 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:15.970457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 552/1105] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:17.017729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dcce9-ca21-4f7d-93d1-ff84515ab6ed None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:17.018338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dcce9-ca21-4f7d-93d1-ff84515ab6ed None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:17.018338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1dcce9-ca21-4f7d-93d1-ff84515ab6ed None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:17.019659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 554/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:17.024491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99377c7-5856-454d-8eee-eb0f52217296 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:17.024933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99377c7-5856-454d-8eee-eb0f52217296 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:17.025038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99377c7-5856-454d-8eee-eb0f52217296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:17.103549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c99377c7-5856-454d-8eee-eb0f52217296 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:17.104739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c99377c7-5856-454d-8eee-eb0f52217296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:17.201790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 553/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:17.359590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1dcce9-ca21-4f7d-93d1-ff84515ab6ed None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:17.361110 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2e1dcce9-ca21-4f7d-93d1-ff84515ab6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:17.410152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:17.508832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1dcce9-ca21-4f7d-93d1-ff84515ab6ed None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:17.620107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cb11d-3dca-42e1-a62b-c589d18eaa93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:17.620715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cb11d-3dca-42e1-a62b-c589d18eaa93 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:17.620999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cb11d-3dca-42e1-a62b-c589d18eaa93 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:17.621274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cb11d-3dca-42e1-a62b-c589d18eaa93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:17.623468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9a0cb11d-3dca-42e1-a62b-c589d18eaa93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:17.630618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:17.712851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cb11d-3dca-42e1-a62b-c589d18eaa93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:17.714483 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cb11d-3dca-42e1-a62b-c589d18eaa93 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:17.733715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 555/1108] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:18.769808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b562f5-b563-4996-8916-87117b3ef6c5 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:18.770373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b562f5-b563-4996-8916-87117b3ef6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:18.770373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b562f5-b563-4996-8916-87117b3ef6c5 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:18.771738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 554/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:06:18.776601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a0f2-1b3e-4b89-84cf-cddf358d9474 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:18.776983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a0f2-1b3e-4b89-84cf-cddf358d9474 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:18.777147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7a0f2-1b3e-4b89-84cf-cddf358d9474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:18.852189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-82e7a0f2-1b3e-4b89-84cf-cddf358d9474 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:18.853376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-82e7a0f2-1b3e-4b89-84cf-cddf358d9474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:18.921114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:18.950321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 556/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:06:19.101036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b562f5-b563-4996-8916-87117b3ef6c5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:19.102587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-97b562f5-b563-4996-8916-87117b3ef6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:19.120919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:19.250085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b562f5-b563-4996-8916-87117b3ef6c5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:19.338685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:19.365270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a32258-53f6-427a-9d53-c9b62a31c28e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:19.365919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a32258-53f6-427a-9d53-c9b62a31c28e service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:19.366206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a32258-53f6-427a-9d53-c9b62a31c28e service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:19.366485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a32258-53f6-427a-9d53-c9b62a31c28e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:19.368135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-95a32258-53f6-427a-9d53-c9b62a31c28e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:19.458201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a32258-53f6-427a-9d53-c9b62a31c28e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:19.459940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a32258-53f6-427a-9d53-c9b62a31c28e service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:19.479561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 555/1111] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:20.534103 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89e87b-2d92-478f-9a90-b6389f67e247 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:20.534634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89e87b-2d92-478f-9a90-b6389f67e247 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:20.534762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89e87b-2d92-478f-9a90-b6389f67e247 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:20.536039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 557/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:20.540970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e53be0-46cd-47a0-8d3e-e0f0c6b70dfc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:20.541411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e53be0-46cd-47a0-8d3e-e0f0c6b70dfc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:20.541507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e53be0-46cd-47a0-8d3e-e0f0c6b70dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:20.618105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-50e53be0-46cd-47a0-8d3e-e0f0c6b70dfc None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:20.619248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-50e53be0-46cd-47a0-8d3e-e0f0c6b70dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:20.660752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:20.717220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 556/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:06:20.871983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89e87b-2d92-478f-9a90-b6389f67e247 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:20.873972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1d89e87b-2d92-478f-9a90-b6389f67e247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:21.020852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89e87b-2d92-478f-9a90-b6389f67e247 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:21.079923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:21.141219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09991f-6018-4a05-abdd-d7233836489f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:21.141754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09991f-6018-4a05-abdd-d7233836489f service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:21.142033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09991f-6018-4a05-abdd-d7233836489f service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:21.142306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09991f-6018-4a05-abdd-d7233836489f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:21.143937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ab09991f-6018-4a05-abdd-d7233836489f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:21.232633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09991f-6018-4a05-abdd-d7233836489f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:21.234217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09991f-6018-4a05-abdd-d7233836489f service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:21.252124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 558/1114] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:22.312129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f77f8-ce66-46bf-b5a5-ace94234e42c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:22.312688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f77f8-ce66-46bf-b5a5-ace94234e42c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:22.312688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f77f8-ce66-46bf-b5a5-ace94234e42c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:22.314018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 557/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:22.319021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de797ac-0dcb-4813-9a63-bc59751d8d02 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:22.319509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de797ac-0dcb-4813-9a63-bc59751d8d02 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:22.319614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de797ac-0dcb-4813-9a63-bc59751d8d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:22.397049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0de797ac-0dcb-4813-9a63-bc59751d8d02 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:22.398162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0de797ac-0dcb-4813-9a63-bc59751d8d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:22.400228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:22.494400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 559/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:22.649224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f77f8-ce66-46bf-b5a5-ace94234e42c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:22.650655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-018f77f8-ce66-46bf-b5a5-ace94234e42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:22.794846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f77f8-ce66-46bf-b5a5-ace94234e42c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:22.821526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:22.906830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b5f57-0f12-4bad-bfff-5c25a0a9baea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:22.907464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b5f57-0f12-4bad-bfff-5c25a0a9baea service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:22.907744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b5f57-0f12-4bad-bfff-5c25a0a9baea service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:22.908013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b5f57-0f12-4bad-bfff-5c25a0a9baea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:22.909594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-376b5f57-0f12-4bad-bfff-5c25a0a9baea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:22.998957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b5f57-0f12-4bad-bfff-5c25a0a9baea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:23.000579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b5f57-0f12-4bad-bfff-5c25a0a9baea service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:23.019150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 558/1117] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:25.091547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f476-5a33-4d5a-85f4-adc207c4d79d None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:25.092040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f476-5a33-4d5a-85f4-adc207c4d79d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:25.092040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f476-5a33-4d5a-85f4-adc207c4d79d None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:25.093381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 560/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:25.098886 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28a20-8ea3-4298-8d60-43435ff80057 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:25.099317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28a20-8ea3-4298-8d60-43435ff80057 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:25.099490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28a20-8ea3-4298-8d60-43435ff80057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:25.141231 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:25.167876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-aed28a20-8ea3-4298-8d60-43435ff80057 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:25.169086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-aed28a20-8ea3-4298-8d60-43435ff80057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:25.265101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 559/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:25.549689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0f476-5a33-4d5a-85f4-adc207c4d79d None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:25.551263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ace0f476-5a33-4d5a-85f4-adc207c4d79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:25.655226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0f476-5a33-4d5a-85f4-adc207c4d79d None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:25.688260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:25.731330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad80e7-a12a-4b56-a0de-b618457fea20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:25.732555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad80e7-a12a-4b56-a0de-b618457fea20 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:25.732851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad80e7-a12a-4b56-a0de-b618457fea20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:25.733131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad80e7-a12a-4b56-a0de-b618457fea20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:25.734727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-78ad80e7-a12a-4b56-a0de-b618457fea20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:25.844121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad80e7-a12a-4b56-a0de-b618457fea20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:25.845480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad80e7-a12a-4b56-a0de-b618457fea20 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:25.863085 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 561/1120] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:28.172562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ef49-1a2b-490c-9a58-bdcced94329d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:28.173120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ef49-1a2b-490c-9a58-bdcced94329d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:28.173120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12ef49-1a2b-490c-9a58-bdcced94329d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:28.174433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 560/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:28.179750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4146a83-bda1-44e2-9cd2-1ab8b33abfde None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:28.180159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4146a83-bda1-44e2-9cd2-1ab8b33abfde None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:28.180307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4146a83-bda1-44e2-9cd2-1ab8b33abfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:28.245826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d4146a83-bda1-44e2-9cd2-1ab8b33abfde None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:28.247029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d4146a83-bda1-44e2-9cd2-1ab8b33abfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:28.260061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:28.346310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 562/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:28.632075 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ef49-1a2b-490c-9a58-bdcced94329d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:28.633591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ed12ef49-1a2b-490c-9a58-bdcced94329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:28.740589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed12ef49-1a2b-490c-9a58-bdcced94329d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:28.754152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:28.817772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d27a46-c00f-45de-90c3-cf61f01c9643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:28.818382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27a46-c00f-45de-90c3-cf61f01c9643 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:28.818663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27a46-c00f-45de-90c3-cf61f01c9643 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:28.818937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27a46-c00f-45de-90c3-cf61f01c9643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:28.820561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-51d27a46-c00f-45de-90c3-cf61f01c9643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:28.929563 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d27a46-c00f-45de-90c3-cf61f01c9643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:28.931021 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d27a46-c00f-45de-90c3-cf61f01c9643 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:28.949365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 561/1123] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:31.461748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e530-36c3-47f5-9390-d687d0ed5e00 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:31.462278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e530-36c3-47f5-9390-d687d0ed5e00 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:31.462278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e530-36c3-47f5-9390-d687d0ed5e00 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:31.463734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 563/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:31.469166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895ce30-9036-4e8f-a95b-f70d1a1affea None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:31.469576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895ce30-9036-4e8f-a95b-f70d1a1affea None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:31.469730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895ce30-9036-4e8f-a95b-f70d1a1affea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:31.518841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:31.536626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a895ce30-9036-4e8f-a95b-f70d1a1affea None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:31.537812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a895ce30-9036-4e8f-a95b-f70d1a1affea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:31.636107 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 562/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:31.917663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7e530-36c3-47f5-9390-d687d0ed5e00 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:31.919232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f6d7e530-36c3-47f5-9390-d687d0ed5e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:32.022931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7e530-36c3-47f5-9390-d687d0ed5e00 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:32.063995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:32.099556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f2207-c30b-4296-aab7-796fc38fb37b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:32.100168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f2207-c30b-4296-aab7-796fc38fb37b service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:32.100452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f2207-c30b-4296-aab7-796fc38fb37b service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:32.100728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f2207-c30b-4296-aab7-796fc38fb37b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:32.102316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c74f2207-c30b-4296-aab7-796fc38fb37b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:32.213370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f2207-c30b-4296-aab7-796fc38fb37b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:32.214724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f2207-c30b-4296-aab7-796fc38fb37b service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:32.233692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 564/1126] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:34.542376 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8360c35-0a33-4546-bfd6-5c8431ff9def None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:34.542948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8360c35-0a33-4546-bfd6-5c8431ff9def None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:34.543235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8360c35-0a33-4546-bfd6-5c8431ff9def None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:34.544717 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 563/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:34.550193 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b6c37-977b-4976-b31b-19ae239b3237 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:34.550589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b6c37-977b-4976-b31b-19ae239b3237 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:34.550785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b6c37-977b-4976-b31b-19ae239b3237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:34.619504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-8c6b6c37-977b-4976-b31b-19ae239b3237 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:34.620712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8c6b6c37-977b-4976-b31b-19ae239b3237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:34.642945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:34.723088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 565/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:35.007852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8360c35-0a33-4546-bfd6-5c8431ff9def None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:35.009380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e8360c35-0a33-4546-bfd6-5c8431ff9def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:35.113784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8360c35-0a33-4546-bfd6-5c8431ff9def None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:35.134780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:35.189794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedec349-9dc1-4dfb-9d30-f18ae90a8fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:35.190398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedec349-9dc1-4dfb-9d30-f18ae90a8fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:35.190679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedec349-9dc1-4dfb-9d30-f18ae90a8fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:35.190951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedec349-9dc1-4dfb-9d30-f18ae90a8fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:35.192567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-eedec349-9dc1-4dfb-9d30-f18ae90a8fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:35.302173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedec349-9dc1-4dfb-9d30-f18ae90a8fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:35.303563 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedec349-9dc1-4dfb-9d30-f18ae90a8fc2 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:35.321872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 564/1129] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:37.540873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c690ff-6735-4b01-a6c3-aef014384b39 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:37.541458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c690ff-6735-4b01-a6c3-aef014384b39 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:37.541458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c690ff-6735-4b01-a6c3-aef014384b39 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:37.542762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 566/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:37.548598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b84f67-d9c7-4273-9f7e-c4d08f062cfb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:37.548992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b84f67-d9c7-4273-9f7e-c4d08f062cfb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:37.549166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b84f67-d9c7-4273-9f7e-c4d08f062cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:37.613780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:37.617113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a5b84f67-d9c7-4273-9f7e-c4d08f062cfb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:37.618891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a5b84f67-d9c7-4273-9f7e-c4d08f062cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:37.716410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 565/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:06:38.006241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c690ff-6735-4b01-a6c3-aef014384b39 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:38.007817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-02c690ff-6735-4b01-a6c3-aef014384b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:38.110854 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c690ff-6735-4b01-a6c3-aef014384b39 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:38.160928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:38.185493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc530c71-e3c3-46bf-9d2c-8d69e9e6dc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:38.186096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530c71-e3c3-46bf-9d2c-8d69e9e6dc7c service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:38.186377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530c71-e3c3-46bf-9d2c-8d69e9e6dc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:38.186647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530c71-e3c3-46bf-9d2c-8d69e9e6dc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:38.188279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-dc530c71-e3c3-46bf-9d2c-8d69e9e6dc7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:38.297003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc530c71-e3c3-46bf-9d2c-8d69e9e6dc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:38.298367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc530c71-e3c3-46bf-9d2c-8d69e9e6dc7c service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:38.317756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 567/1132] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:40.689700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4274ed-cd18-4223-882f-7e9d519dfff0 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:40.690244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4274ed-cd18-4223-882f-7e9d519dfff0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:40.690244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4274ed-cd18-4223-882f-7e9d519dfff0 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:40.691667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 566/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:40.697130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57c6ce-8edb-456d-8ba6-3483fad57cba None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:40.697553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57c6ce-8edb-456d-8ba6-3483fad57cba None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:40.697685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57c6ce-8edb-456d-8ba6-3483fad57cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:40.765766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7c57c6ce-8edb-456d-8ba6-3483fad57cba None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:40.767005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c57c6ce-8edb-456d-8ba6-3483fad57cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:40.796575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:40.865638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 568/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:41.156653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4274ed-cd18-4223-882f-7e9d519dfff0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:41.158568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5b4274ed-cd18-4223-882f-7e9d519dfff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:41.262828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4274ed-cd18-4223-882f-7e9d519dfff0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:41.291325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:41.337612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13876f10-b373-4460-b7e9-5e4540128cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:41.338247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13876f10-b373-4460-b7e9-5e4540128cae service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:41.338523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13876f10-b373-4460-b7e9-5e4540128cae service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:41.338791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13876f10-b373-4460-b7e9-5e4540128cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:41.340421 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-13876f10-b373-4460-b7e9-5e4540128cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:41.448104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13876f10-b373-4460-b7e9-5e4540128cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:41.449540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13876f10-b373-4460-b7e9-5e4540128cae service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:41.467504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 567/1135] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:43.686560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad9683-35db-4da3-a612-1bb3584b275d None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:43.687253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad9683-35db-4da3-a612-1bb3584b275d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:43.687253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad9683-35db-4da3-a612-1bb3584b275d None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:43.688487 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 569/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:43.693815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3e23-f789-4e82-94db-aff28102324e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:43.694276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3e23-f789-4e82-94db-aff28102324e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:43.694386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd3e23-f789-4e82-94db-aff28102324e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:43.760978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-62cd3e23-f789-4e82-94db-aff28102324e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:43.762180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-62cd3e23-f789-4e82-94db-aff28102324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:43.770560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:43.859841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 568/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:44.154683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad9683-35db-4da3-a612-1bb3584b275d None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:44.156302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b6ad9683-35db-4da3-a612-1bb3584b275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:44.260395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad9683-35db-4da3-a612-1bb3584b275d None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:44.320314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:44.345407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae428af3-5929-435e-8f94-d2c2593fdf59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:44.345925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae428af3-5929-435e-8f94-d2c2593fdf59 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:44.346207 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae428af3-5929-435e-8f94-d2c2593fdf59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:44.346481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae428af3-5929-435e-8f94-d2c2593fdf59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:44.348136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ae428af3-5929-435e-8f94-d2c2593fdf59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:44.457309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae428af3-5929-435e-8f94-d2c2593fdf59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:44.458746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae428af3-5929-435e-8f94-d2c2593fdf59 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:44.477364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 570/1138] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:46.663308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b9eb-547f-4cf4-af8f-d00a3209d000 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:46.663857 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b9eb-547f-4cf4-af8f-d00a3209d000 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:46.663857 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5b9eb-547f-4cf4-af8f-d00a3209d000 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:46.665258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 569/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:46.671125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147248e-75b2-4d9d-afe7-8b37fb2454f4 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:46.671565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147248e-75b2-4d9d-afe7-8b37fb2454f4 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:46.671679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147248e-75b2-4d9d-afe7-8b37fb2454f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:46.738398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c147248e-75b2-4d9d-afe7-8b37fb2454f4 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:46.739628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c147248e-75b2-4d9d-afe7-8b37fb2454f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:46.777212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:46.834850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 571/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:47.115981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5b9eb-547f-4cf4-af8f-d00a3209d000 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:47.117515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-96a5b9eb-547f-4cf4-af8f-d00a3209d000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:47.224216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5b9eb-547f-4cf4-af8f-d00a3209d000 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:47.265965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:47.301533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c3eeb1-f8db-48a8-a3df-4f06aadc00c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:47.302136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3eeb1-f8db-48a8-a3df-4f06aadc00c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:47.302413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3eeb1-f8db-48a8-a3df-4f06aadc00c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:47.302696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c3eeb1-f8db-48a8-a3df-4f06aadc00c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:47.304300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c1c3eeb1-f8db-48a8-a3df-4f06aadc00c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:47.411791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3eeb1-f8db-48a8-a3df-4f06aadc00c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:47.413228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c3eeb1-f8db-48a8-a3df-4f06aadc00c9 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:47.431460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 570/1141] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:49.587555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99dbb-d0d1-4541-817c-21fb3743fe04 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:49.588138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99dbb-d0d1-4541-817c-21fb3743fe04 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:49.588138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a99dbb-d0d1-4541-817c-21fb3743fe04 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:49.589458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 572/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:49.595280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c06b-3ac0-4e0a-82df-cb6daa1d9026 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:49.595694 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c06b-3ac0-4e0a-82df-cb6daa1d9026 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:49.595846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663c06b-3ac0-4e0a-82df-cb6daa1d9026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:49.662292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1663c06b-3ac0-4e0a-82df-cb6daa1d9026 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:49.663553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1663c06b-3ac0-4e0a-82df-cb6daa1d9026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:49.681832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:49.764567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 571/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:50.048231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99dbb-d0d1-4541-817c-21fb3743fe04 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:50.049900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e6a99dbb-d0d1-4541-817c-21fb3743fe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:50.159590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a99dbb-d0d1-4541-817c-21fb3743fe04 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:50.226041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:50.232790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bff9a-e708-4576-b643-5793ac56a774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:50.233401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bff9a-e708-4576-b643-5793ac56a774 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:50.233687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bff9a-e708-4576-b643-5793ac56a774 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:50.233965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bff9a-e708-4576-b643-5793ac56a774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:50.235594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e64bff9a-e708-4576-b643-5793ac56a774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:50.349025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bff9a-e708-4576-b643-5793ac56a774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:50.350366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bff9a-e708-4576-b643-5793ac56a774 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:50.368840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 573/1144] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:06:52.698935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b908b-eb3d-4de9-83bd-9dc2c3862bee None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:52.699530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b908b-eb3d-4de9-83bd-9dc2c3862bee None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:52.699530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b908b-eb3d-4de9-83bd-9dc2c3862bee None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:52.701590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 572/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:06:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:06:52.706947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7ec3-9c98-4efb-b95a-bb5e9c1fe126 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:52.707407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7ec3-9c98-4efb-b95a-bb5e9c1fe126 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:52.707549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7ec3-9c98-4efb-b95a-bb5e9c1fe126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:52.774442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-862b7ec3-9c98-4efb-b95a-bb5e9c1fe126 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:06:52.775658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-862b7ec3-9c98-4efb-b95a-bb5e9c1fe126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:52.822513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:52.870631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 574/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:06:53.156037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b908b-eb3d-4de9-83bd-9dc2c3862bee None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:06:53.157595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-682b908b-eb3d-4de9-83bd-9dc2c3862bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:53.262470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b908b-eb3d-4de9-83bd-9dc2c3862bee None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:06:53.310472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:06:53.336925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f116f-8e96-4cfb-bdad-ca4d366cbf8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:06:53.337534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f116f-8e96-4cfb-bdad-ca4d366cbf8d service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:06:53.337874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f116f-8e96-4cfb-bdad-ca4d366cbf8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:06:53.338158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f116f-8e96-4cfb-bdad-ca4d366cbf8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:06:53.339815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-eb4f116f-8e96-4cfb-bdad-ca4d366cbf8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:06:53.449039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f116f-8e96-4cfb-bdad-ca4d366cbf8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:06:53.450534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f116f-8e96-4cfb-bdad-ca4d366cbf8d service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:06:53.468477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 573/1147] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:07:02.869142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=22 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:03.466255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=66 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:13.519021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f440dc-646b-43b8-866d-d8c2658020ff None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.519579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f440dc-646b-43b8-866d-d8c2658020ff None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.519703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f440dc-646b-43b8-866d-d8c2658020ff None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.520962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 575/1148] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:13.526290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d458-c7a8-4334-82b5-f266dea828ad None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.526691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d458-c7a8-4334-82b5-f266dea828ad None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.526845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d458-c7a8-4334-82b5-f266dea828ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.593553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-aa92d458-c7a8-4334-82b5-f266dea828ad None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:13.594739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-aa92d458-c7a8-4334-82b5-f266dea828ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:13.688152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 574/1149] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:07:13.987799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29271950-81a7-412d-a731-e019dc41242d None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.988091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29271950-81a7-412d-a731-e019dc41242d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.988398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29271950-81a7-412d-a731-e019dc41242d None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:13.989766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 576/1150] 10.222.0.42 () {54 vars in 962 bytes} [Mon Oct 24 13:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:13.994683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2894-46af-48f6-bff9-821d0699a314 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:13.994968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2894-46af-48f6-bff9-821d0699a314 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:13.995271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f2894-46af-48f6-bff9-821d0699a314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:14.072260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-447f2894-46af-48f6-bff9-821d0699a314 None None] MFA Rules not processed for user `19d0fc260e254d2d80c6927c52cd8a37`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:14.073470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-447f2894-46af-48f6-bff9-821d0699a314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:14.094528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:14.150075 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 575/1151] 10.222.0.42 () {58 vars in 1059 bytes} [Mon Oct 24 13:07:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:07:14.158477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29271950-81a7-412d-a731-e019dc41242d None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:14.160088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-29271950-81a7-412d-a731-e019dc41242d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:14.268098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29271950-81a7-412d-a731-e019dc41242d None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:14.344592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f35ac-c860-48b9-9722-917ab6420c67 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d0fc260e254d2d80c6927c52cd8a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:14.345211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f35ac-c860-48b9-9722-917ab6420c67 service glance] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:14.345525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f35ac-c860-48b9-9722-917ab6420c67 service glance] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:14.345802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f35ac-c860-48b9-9722-917ab6420c67 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:14.347505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-df3f35ac-c860-48b9-9722-917ab6420c67 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:14.405128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:14.461240 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f35ac-c860-48b9-9722-917ab6420c67 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:14.462708 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f35ac-c860-48b9-9722-917ab6420c67 service glance] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:14.482359 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 577/1152] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:07:16.717015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceef2c-bfc5-410d-8e65-d783b26b6960 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:16.717592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceef2c-bfc5-410d-8e65-d783b26b6960 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:16.717592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dceef2c-bfc5-410d-8e65-d783b26b6960 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:16.718896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 576/1153] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:16.725018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19928fe3-b60e-4238-90c7-91ee34c63001 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:16.725451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19928fe3-b60e-4238-90c7-91ee34c63001 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:16.725566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19928fe3-b60e-4238-90c7-91ee34c63001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:16.791259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-19928fe3-b60e-4238-90c7-91ee34c63001 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:16.792448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-19928fe3-b60e-4238-90c7-91ee34c63001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:16.888875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 578/1154] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:07:17.162993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dceef2c-bfc5-410d-8e65-d783b26b6960 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:17.164565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7dceef2c-bfc5-410d-8e65-d783b26b6960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:17.299798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dceef2c-bfc5-410d-8e65-d783b26b6960 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:17.350441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:17.395788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d21bb2-4fb3-4b04-8198-a9eb28747f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:17.396394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d21bb2-4fb3-4b04-8198-a9eb28747f0b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:17.396669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d21bb2-4fb3-4b04-8198-a9eb28747f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:17.396939 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d21bb2-4fb3-4b04-8198-a9eb28747f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:17.397570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d21bb2-4fb3-4b04-8198-a9eb28747f0b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:17.398576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-59d21bb2-4fb3-4b04-8198-a9eb28747f0b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:07:17.398576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 13:07:17.398576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 13:07:17.399741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d21bb2-4fb3-4b04-8198-a9eb28747f0b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:17.418681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 577/1155] 10.222.0.42 () {56 vars in 1108 bytes} [Mon Oct 24 13:07:17 2022] GET /identity/v3/projects => generated 2553 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:07:19.587325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9de4a-12c9-4ce6-ada2-e45445f4a531 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.587862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9de4a-12c9-4ce6-ada2-e45445f4a531 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.587989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc9de4a-12c9-4ce6-ada2-e45445f4a531 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.589538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 579/1156] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:19.594779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442982f-0a9e-44d2-917e-f469c0b91f70 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:19.595243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442982f-0a9e-44d2-917e-f469c0b91f70 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:19.595372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442982f-0a9e-44d2-917e-f469c0b91f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:19.660866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a442982f-0a9e-44d2-917e-f469c0b91f70 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:19.662048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a442982f-0a9e-44d2-917e-f469c0b91f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:19.758977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 578/1157] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:07:20.050576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9de4a-12c9-4ce6-ada2-e45445f4a531 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:20.052304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1dc9de4a-12c9-4ce6-ada2-e45445f4a531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:20.066042 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:20.202169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc9de4a-12c9-4ce6-ada2-e45445f4a531 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:20.289191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:20.323113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ed2a8-e23b-446a-ac12-debf5f2a1a6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:20.323729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ed2a8-e23b-446a-ac12-debf5f2a1a6c service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:20.324063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ed2a8-e23b-446a-ac12-debf5f2a1a6c service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:20.324326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ed2a8-e23b-446a-ac12-debf5f2a1a6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:20.326000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-747ed2a8-e23b-446a-ac12-debf5f2a1a6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:20.420183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ed2a8-e23b-446a-ac12-debf5f2a1a6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:20.421841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ed2a8-e23b-446a-ac12-debf5f2a1a6c service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:20.440593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 580/1158] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:07:22.784142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9394-0c71-47ab-91a6-500d90554e8f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.784713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9394-0c71-47ab-91a6-500d90554e8f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.784713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c9394-0c71-47ab-91a6-500d90554e8f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.786034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 579/1159] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:07:22.791453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948a1c9-f494-466e-a0eb-fb023f83a9c1 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:22.791896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948a1c9-f494-466e-a0eb-fb023f83a9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:22.792013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948a1c9-f494-466e-a0eb-fb023f83a9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:22.857485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4948a1c9-f494-466e-a0eb-fb023f83a9c1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:22.858689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4948a1c9-f494-466e-a0eb-fb023f83a9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:22.904504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:22.954899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 581/1160] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:07:23.248322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9394-0c71-47ab-91a6-500d90554e8f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:23.249834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-272c9394-0c71-47ab-91a6-500d90554e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:23.274147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.355272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c9394-0c71-47ab-91a6-500d90554e8f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:23.430535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2377c47-4a73-41b5-abd1-99846ef74adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:23.431172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2377c47-4a73-41b5-abd1-99846ef74adf service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:23.431455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2377c47-4a73-41b5-abd1-99846ef74adf service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:23.431728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2377c47-4a73-41b5-abd1-99846ef74adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:23.433324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f2377c47-4a73-41b5-abd1-99846ef74adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:23.507877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:23.542751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2377c47-4a73-41b5-abd1-99846ef74adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:23.544737 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2377c47-4a73-41b5-abd1-99846ef74adf service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:23.564151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 580/1161] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:07:25.787177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146332-ad72-4699-a756-7e7e4b99359f None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:25.787747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146332-ad72-4699-a756-7e7e4b99359f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:25.787747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12146332-ad72-4699-a756-7e7e4b99359f None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:25.789089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 582/1162] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:25.794933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff1be-e2cd-4bc0-b42b-6ecd4ca7b6db None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:25.795428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff1be-e2cd-4bc0-b42b-6ecd4ca7b6db None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:25.795562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ff1be-e2cd-4bc0-b42b-6ecd4ca7b6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:25.861622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1a6ff1be-e2cd-4bc0-b42b-6ecd4ca7b6db None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:25.862843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1a6ff1be-e2cd-4bc0-b42b-6ecd4ca7b6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:25.960029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 581/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:07:26.247034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12146332-ad72-4699-a756-7e7e4b99359f None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:26.248743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-12146332-ad72-4699-a756-7e7e4b99359f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:26.353921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12146332-ad72-4699-a756-7e7e4b99359f None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:26.425685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:26.427182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214703f9-8c29-4714-9cc6-058129bcbfe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:26.427852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214703f9-8c29-4714-9cc6-058129bcbfe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:26.428349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214703f9-8c29-4714-9cc6-058129bcbfe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:26.428795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214703f9-8c29-4714-9cc6-058129bcbfe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:26.430619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-214703f9-8c29-4714-9cc6-058129bcbfe3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:26.538899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214703f9-8c29-4714-9cc6-058129bcbfe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:26.540369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214703f9-8c29-4714-9cc6-058129bcbfe3 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:26.557748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 583/1164] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:07:28.885731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd3d94-6839-486d-ad30-fcb8083baf96 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:28.886761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd3d94-6839-486d-ad30-fcb8083baf96 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:28.887322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd3d94-6839-486d-ad30-fcb8083baf96 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:28.889811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 582/1165] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:07:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:28.895555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601fbf-9bdf-4a2a-b0e2-6c2332da4c84 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:28.896010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601fbf-9bdf-4a2a-b0e2-6c2332da4c84 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:28.896149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601fbf-9bdf-4a2a-b0e2-6c2332da4c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:28.966819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-59601fbf-9bdf-4a2a-b0e2-6c2332da4c84 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:28.968076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-59601fbf-9bdf-4a2a-b0e2-6c2332da4c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:29.025317 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.072251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 584/1166] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:07:29.367986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd3d94-6839-486d-ad30-fcb8083baf96 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:29.369539 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-88cd3d94-6839-486d-ad30-fcb8083baf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:29.406555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.473307 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd3d94-6839-486d-ad30-fcb8083baf96 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:29.547084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5427b-5ef5-46b4-90a6-ede161c0a71a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:29.547705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5427b-5ef5-46b4-90a6-ede161c0a71a service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:29.547986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5427b-5ef5-46b4-90a6-ede161c0a71a service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:29.548259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5427b-5ef5-46b4-90a6-ede161c0a71a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:29.549846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5cc5427b-5ef5-46b4-90a6-ede161c0a71a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:29.631686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:29.661584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5427b-5ef5-46b4-90a6-ede161c0a71a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:29.663255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5427b-5ef5-46b4-90a6-ede161c0a71a service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:29.681364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 583/1167] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:07:32.093026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6467-8cf3-481e-8d2b-8941781998fa None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.093544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6467-8cf3-481e-8d2b-8941781998fa None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.093640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6467-8cf3-481e-8d2b-8941781998fa None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.094961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 585/1168] 10.222.0.42 () {54 vars in 927 bytes} [Mon Oct 24 13:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:07:32.100258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92046ff-605d-4064-81f0-8856f15ab240 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.100674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92046ff-605d-4064-81f0-8856f15ab240 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.100823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92046ff-605d-4064-81f0-8856f15ab240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.167289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c92046ff-605d-4064-81f0-8856f15ab240 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:07:32.168518 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c92046ff-605d-4064-81f0-8856f15ab240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:32.266714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 584/1169] 10.222.0.42 () {58 vars in 1024 bytes} [Mon Oct 24 13:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:07:32.558181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f6467-8cf3-481e-8d2b-8941781998fa None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:07:32.559841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-902f6467-8cf3-481e-8d2b-8941781998fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:32.664668 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f6467-8cf3-481e-8d2b-8941781998fa None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:07:32.742460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f296d4-3462-4102-ae5d-fe4c027960c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:07:32.743097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f296d4-3462-4102-ae5d-fe4c027960c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:07:32.743383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f296d4-3462-4102-ae5d-fe4c027960c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:07:32.743664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f296d4-3462-4102-ae5d-fe4c027960c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:07:32.745263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d3f296d4-3462-4102-ae5d-fe4c027960c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:07:32.754571 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:32.855373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f296d4-3462-4102-ae5d-fe4c027960c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:07:32.856921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f296d4-3462-4102-ae5d-fe4c027960c5 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:07:32.875095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 586/1170] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:07:42.265211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=82 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:07:42.872467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=50 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:47.283927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eeb7a-a5cf-4994-8555-40818b02e600 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.284453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eeb7a-a5cf-4994-8555-40818b02e600 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.284453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eeb7a-a5cf-4994-8555-40818b02e600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.360631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-406eeb7a-a5cf-4994-8555-40818b02e600 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:47.361845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-406eeb7a-a5cf-4994-8555-40818b02e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:47.457581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 585/1171] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:08:47.466560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d208af-55ed-4f13-b801-a29cdf1525d7 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.467041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d208af-55ed-4f13-b801-a29cdf1525d7 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.467179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d208af-55ed-4f13-b801-a29cdf1525d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.545180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-b5d208af-55ed-4f13-b801-a29cdf1525d7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:47.546283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b5d208af-55ed-4f13-b801-a29cdf1525d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:47.642927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 587/1172] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:08:47.647543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eeb7a-a5cf-4994-8555-40818b02e600 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:47.648751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-406eeb7a-a5cf-4994-8555-40818b02e600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:47.660958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eeb7a-a5cf-4994-8555-40818b02e600 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:47.722341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:47.756592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:47.757223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.757752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.758022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.758659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:08:47.758959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:47.759995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:08:47.759995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 13:08:47.759995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 13:08:47.760875 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:47.772458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 586/1173] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:08:47.782081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d208af-55ed-4f13-b801-a29cdf1525d7 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:47.791153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d208af-55ed-4f13-b801-a29cdf1525d7 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:47.851921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:47.886056 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:47.887271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:47.887555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:47.887822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:47.888452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:47.889552 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING py.warnings [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:08:47.889552 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: warnings.warn(msg) Oct 24 13:08:47.889552 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: Oct 24 13:08:47.890769 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:47.946104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 588/1174] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:08:47 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:47.950535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:47.952122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:47.986149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feed9e-d3bf-40a2-9e0d-1135def2e44f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:48.079681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:48.080305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.080589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.080862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.081510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:48.082521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:08:48.082521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 13:08:48.082521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 13:08:48.083768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:48.125503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 587/1175] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:08:47 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:08:48.129555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:48.131074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:48.192619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1473e3b-0203-4634-8c7b-2381642768d7 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:48.277846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:48.289357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:48.289969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.290249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.290524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.291196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:48.293494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:48.305785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 589/1176] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:08:48.310410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:48.311984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:48.323419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12831ea5-6a20-48c7-bf29-14035c79e1b7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:48.348596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:48.422610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1144926-16c6-45ea-8d5c-1f74511b8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:48.423260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1144926-16c6-45ea-8d5c-1f74511b8148 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.423541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1144926-16c6-45ea-8d5c-1f74511b8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.423813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1144926-16c6-45ea-8d5c-1f74511b8148 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.424451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1144926-16c6-45ea-8d5c-1f74511b8148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:48.426037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1144926-16c6-45ea-8d5c-1f74511b8148 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:48.438817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 588/1177] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:08:48.442699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:48.444290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:48.477892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc81b73-c81b-4f01-bcff-5d0ea558aacb admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:48.572094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00644321-3801-4578-b653-1a6f6d2d5cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:48.572747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00644321-3801-4578-b653-1a6f6d2d5cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.573027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00644321-3801-4578-b653-1a6f6d2d5cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.573297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00644321-3801-4578-b653-1a6f6d2d5cf4 admin admin] PATH_INFO: `/v3/projects/e3244ec565c74b29ad216ca6fd3f7d7b/users/7a6c654c84844a42adddb012ae0aa85c/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.596245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00644321-3801-4578-b653-1a6f6d2d5cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3244ec565c74b29ad216ca6fd3f7d7b, user_id=7a6c654c84844a42adddb012ae0aa85c, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:48.598882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00644321-3801-4578-b653-1a6f6d2d5cf4 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:48.623980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 590/1178] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:08:48 2022] PUT /identity/v3/projects/e3244ec565c74b29ad216ca6fd3f7d7b/users/7a6c654c84844a42adddb012ae0aa85c/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:08:48.635342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16549773-af54-4b3f-b922-ca86ab5d3f7b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.635636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16549773-af54-4b3f-b922-ca86ab5d3f7b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.635916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16549773-af54-4b3f-b922-ca86ab5d3f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.693712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-16549773-af54-4b3f-b922-ca86ab5d3f7b None None] MFA Rules not processed for user `7a6c654c84844a42adddb012ae0aa85c`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:48.694861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-16549773-af54-4b3f-b922-ca86ab5d3f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:48.773370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 589/1179] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:08:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:08:48.787364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd2b0-2c61-4795-9b59-345fbf658c32 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.787663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd2b0-2c61-4795-9b59-345fbf658c32 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.787943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecd2b0-2c61-4795-9b59-345fbf658c32 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.789312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 591/1180] 10.222.0.42 () {54 vars in 963 bytes} [Mon Oct 24 13:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:48.794090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d9b7c-0279-445d-a7c1-4ba6747deceb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:48.794377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d9b7c-0279-445d-a7c1-4ba6747deceb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:48.794662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d9b7c-0279-445d-a7c1-4ba6747deceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:48.826377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:48.870487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-594d9b7c-0279-445d-a7c1-4ba6747deceb None None] MFA Rules not processed for user `fee3f4d7b69f4851b9160f68a3937703`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:48.871769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-594d9b7c-0279-445d-a7c1-4ba6747deceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:48.946826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 590/1181] 10.222.0.42 () {58 vars in 1060 bytes} [Mon Oct 24 13:08:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:08:48.955297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd2b0-2c61-4795-9b59-345fbf658c32 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:48.956834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-13ecd2b0-2c61-4795-9b59-345fbf658c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:49.006720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.062385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecd2b0-2c61-4795-9b59-345fbf658c32 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.138455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.139083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.139367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.139640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.141208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:49.241389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.242748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.246739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.262448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 592/1182] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:08:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:08:49.306935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f58a80-56c4-4022-843d-61a88e80585f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.307259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f58a80-56c4-4022-843d-61a88e80585f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.307543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f58a80-56c4-4022-843d-61a88e80585f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.309377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 591/1183] 10.222.0.42 () {52 vars in 830 bytes} [Mon Oct 24 13:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:08:49.312964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.314169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:49.413346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1493-7f5c-4cea-a77f-7c4461115bc1 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.506627 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.507283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.507735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.508004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] PATH_INFO: `/v3/users/7a6c654c84844a42adddb012ae0aa85c` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.521668 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6c654c84844a42adddb012ae0aa85c)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:49.522674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING py.warnings [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:08:49.522674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: warnings.warn(msg) Oct 24 13:08:49.522674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: Oct 24 13:08:49.523908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:49.534078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.592116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6c654c84844a42adddb012ae0aa85c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:49.615570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6c654c84844a42adddb012ae0aa85c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:49.616078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6c654c84844a42adddb012ae0aa85c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:49.626254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6c654c84844a42adddb012ae0aa85c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:49.640348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6c654c84844a42adddb012ae0aa85c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:49.646249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 593/1184] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:08:49 2022] DELETE /identity/v3/users/7a6c654c84844a42adddb012ae0aa85c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:08:49.650655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9ca73-6665-4456-aa8f-274ee03e60c7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.651148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9ca73-6665-4456-aa8f-274ee03e60c7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.651252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9ca73-6665-4456-aa8f-274ee03e60c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.709610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-abc9ca73-6665-4456-aa8f-274ee03e60c7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:08:49.710816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-abc9ca73-6665-4456-aa8f-274ee03e60c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:49.786900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.800192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 592/1185] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:08:49.811137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:49.812386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:49.913175 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a850bcf-d168-4b64-981f-5237bcb189fb admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:49.967766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:08:49.989570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761f470-4d62-4a66-970f-09662db1c868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:49.990187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761f470-4d62-4a66-970f-09662db1c868 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:49.990469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761f470-4d62-4a66-970f-09662db1c868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:49.990743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761f470-4d62-4a66-970f-09662db1c868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:49.992370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f761f470-4d62-4a66-970f-09662db1c868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:50.103452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761f470-4d62-4a66-970f-09662db1c868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.104900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761f470-4d62-4a66-970f-09662db1c868 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.123705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 594/1186] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:08:50.471306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9ca73-6665-4456-aa8f-274ee03e60c7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:08:50.472563 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-abc9ca73-6665-4456-aa8f-274ee03e60c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:08:50.484839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9ca73-6665-4456-aa8f-274ee03e60c7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:08:50.583394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:08:50.584056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:08:50.584338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:08:50.584610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] PATH_INFO: `/v3/projects/e3244ec565c74b29ad216ca6fd3f7d7b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:08:50.596348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3244ec565c74b29ad216ca6fd3f7d7b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:08:50.597302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:08:50.597302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 13:08:50.597302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 13:08:50.598517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:08:50.657096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3244ec565c74b29ad216ca6fd3f7d7b', 'request_id': 'req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:50.670390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3244ec565c74b29ad216ca6fd3f7d7b', 'request_id': 'req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:50.680297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3244ec565c74b29ad216ca6fd3f7d7b', 'request_id': 'req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:50.704357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] The token cache is being invalidate because project e3244ec565c74b29ad216ca6fd3f7d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:08:50.704650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:50.705078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3f2b66d3-a32a-4aff-a872-5e9acd42c88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:08:50.706466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 593/1187] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:08:50 2022] DELETE /identity/v3/projects/e3244ec565c74b29ad216ca6fd3f7d7b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:09:00.121181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=62 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:09:00.702382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:20.444447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7eda0-8894-40a0-b446-f7d027ee2081 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:20.445058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7eda0-8894-40a0-b446-f7d027ee2081 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:20.445058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7eda0-8894-40a0-b446-f7d027ee2081 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:20.446422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 595/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:20.451364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb98b17-33ac-4889-a9d8-a90cef63513d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:20.451800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb98b17-33ac-4889-a9d8-a90cef63513d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:20.451904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb98b17-33ac-4889-a9d8-a90cef63513d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:20.532037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3eb98b17-33ac-4889-a9d8-a90cef63513d None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:20.533250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3eb98b17-33ac-4889-a9d8-a90cef63513d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:20.637510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 594/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:20.794105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7eda0-8894-40a0-b446-f7d027ee2081 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:20.796355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-71d7eda0-8894-40a0-b446-f7d027ee2081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:20.920752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7eda0-8894-40a0-b446-f7d027ee2081 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:20.926382 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1154c5-bc56-4aac-af23-06dc69d1c7ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:20.927006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1154c5-bc56-4aac-af23-06dc69d1c7ef service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:20.927336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1154c5-bc56-4aac-af23-06dc69d1c7ef service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:20.927612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1154c5-bc56-4aac-af23-06dc69d1c7ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:20.936413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1154c5-bc56-4aac-af23-06dc69d1c7ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:20.938007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1154c5-bc56-4aac-af23-06dc69d1c7ef service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:20.947152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 596/1190] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:22.035562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8539bee-c5d8-41b6-a947-28f22d824a5a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:22.036129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8539bee-c5d8-41b6-a947-28f22d824a5a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:22.036129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8539bee-c5d8-41b6-a947-28f22d824a5a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:22.037462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 595/1191] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:22.042350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2822b9-5ce8-4d02-afe9-2ba3cb460963 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:22.042750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2822b9-5ce8-4d02-afe9-2ba3cb460963 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:22.042901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2822b9-5ce8-4d02-afe9-2ba3cb460963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:22.060932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca377677-9f97-45f2-8c85-00a44271dd7c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:22.061221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca377677-9f97-45f2-8c85-00a44271dd7c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:22.061501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca377677-9f97-45f2-8c85-00a44271dd7c None None] PATH_INFO: `/v3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:22.062060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-8b2822b9-5ce8-4d02-afe9-2ba3cb460963 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:22.062970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 596/1192] 10.222.0.25 () {54 vars in 925 bytes} [Mon Oct 24 13:10:22 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:10:22.063301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8b2822b9-5ce8-4d02-afe9-2ba3cb460963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:22.068450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79775de-4586-4b0c-aae9-c4ccdc18c871 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:22.068731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79775de-4586-4b0c-aae9-c4ccdc18c871 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:22.069009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79775de-4586-4b0c-aae9-c4ccdc18c871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:22.071380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 597/1193] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:22.134645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a79775de-4586-4b0c-aae9-c4ccdc18c871 None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:22.136711 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a79775de-4586-4b0c-aae9-c4ccdc18c871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:22.167819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:22.211572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 597/1194] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Oct 24 13:10:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:22.223867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2822b9-5ce8-4d02-afe9-2ba3cb460963 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:22.236463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2822b9-5ce8-4d02-afe9-2ba3cb460963 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:22.241912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dc743-5e0e-4e79-85ea-03460248e6bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:22.242508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dc743-5e0e-4e79-85ea-03460248e6bf service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:22.243026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dc743-5e0e-4e79-85ea-03460248e6bf service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:22.243343 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dc743-5e0e-4e79-85ea-03460248e6bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:22.252523 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dc743-5e0e-4e79-85ea-03460248e6bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:22.254170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dc743-5e0e-4e79-85ea-03460248e6bf service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:22.264258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 598/1195] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:25.684120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c19546-881b-4571-bfcc-b6cb9033009e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:25.684596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c19546-881b-4571-bfcc-b6cb9033009e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:25.684696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c19546-881b-4571-bfcc-b6cb9033009e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:25.686022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 598/1196] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 24 13:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:25.690867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c854055-e7cd-427a-9ee5-94962563e739 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:25.691297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c854055-e7cd-427a-9ee5-94962563e739 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:25.691458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c854055-e7cd-427a-9ee5-94962563e739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:25.711325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-8c854055-e7cd-427a-9ee5-94962563e739 None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:25.712504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8c854055-e7cd-427a-9ee5-94962563e739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:25.721836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 599/1197] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 24 13:10:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:10:25.729470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c19546-881b-4571-bfcc-b6cb9033009e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:25.743432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c19546-881b-4571-bfcc-b6cb9033009e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:25.748532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4e8ac-8ef6-4ad1-a64b-75dc59b8be88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:25.749154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4e8ac-8ef6-4ad1-a64b-75dc59b8be88 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:25.749441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4e8ac-8ef6-4ad1-a64b-75dc59b8be88 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:25.749716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4e8ac-8ef6-4ad1-a64b-75dc59b8be88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:25.758005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4e8ac-8ef6-4ad1-a64b-75dc59b8be88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:25.759894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4e8ac-8ef6-4ad1-a64b-75dc59b8be88 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:25.771230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 599/1198] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:33.717867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dfc1c-6316-4324-8538-b1b2d8e1a944 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:33.718435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dfc1c-6316-4324-8538-b1b2d8e1a944 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:33.718435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dfc1c-6316-4324-8538-b1b2d8e1a944 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:33.719820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 600/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:33.724486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a424d83-6b7a-413b-a571-b40ded662025 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:33.724890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a424d83-6b7a-413b-a571-b40ded662025 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:33.725044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a424d83-6b7a-413b-a571-b40ded662025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:33.745785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5a424d83-6b7a-413b-a571-b40ded662025 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:33.747510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5a424d83-6b7a-413b-a571-b40ded662025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:33.755320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 600/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:33.908939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dfc1c-6316-4324-8538-b1b2d8e1a944 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:33.923271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dfc1c-6316-4324-8538-b1b2d8e1a944 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:33.928561 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54003b40-bc33-44d0-b617-387cd6ea41ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:33.929154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54003b40-bc33-44d0-b617-387cd6ea41ea service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:33.929433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54003b40-bc33-44d0-b617-387cd6ea41ea service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:33.929709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54003b40-bc33-44d0-b617-387cd6ea41ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:33.938357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54003b40-bc33-44d0-b617-387cd6ea41ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:33.939967 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54003b40-bc33-44d0-b617-387cd6ea41ea service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:33.949025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 601/1201] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:35.016604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53ce8-6c7a-479e-8e8b-074094059dcb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:35.017185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53ce8-6c7a-479e-8e8b-074094059dcb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:35.017185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e53ce8-6c7a-479e-8e8b-074094059dcb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:35.018528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 601/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 24 13:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:35.023364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2721fc-b4b0-45a1-aec3-f569d7ba0600 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:35.023770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2721fc-b4b0-45a1-aec3-f569d7ba0600 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:35.023927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2721fc-b4b0-45a1-aec3-f569d7ba0600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:35.043322 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-af2721fc-b4b0-45a1-aec3-f569d7ba0600 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:35.045161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-af2721fc-b4b0-45a1-aec3-f569d7ba0600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:35.053150 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 602/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 24 13:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:35.205516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e53ce8-6c7a-479e-8e8b-074094059dcb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:35.220176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e53ce8-6c7a-479e-8e8b-074094059dcb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:35.225487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1b8c6b-a97a-4fc7-98b5-9517c33b9739 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:35.226081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b8c6b-a97a-4fc7-98b5-9517c33b9739 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:35.226364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b8c6b-a97a-4fc7-98b5-9517c33b9739 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:35.226641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1b8c6b-a97a-4fc7-98b5-9517c33b9739 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:35.235543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b8c6b-a97a-4fc7-98b5-9517c33b9739 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:35.237159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1b8c6b-a97a-4fc7-98b5-9517c33b9739 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:35.246999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 602/1204] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:39.288244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0242-653b-4b32-a0bd-5d9649c294f7 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:39.288829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0242-653b-4b32-a0bd-5d9649c294f7 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:39.288829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035f0242-653b-4b32-a0bd-5d9649c294f7 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:39.290163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 603/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:39.296547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8f8b6-6226-4cf7-93e2-ec508e0cf013 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:39.297011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8f8b6-6226-4cf7-93e2-ec508e0cf013 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:39.297116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8f8b6-6226-4cf7-93e2-ec508e0cf013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:39.316419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-22d8f8b6-6226-4cf7-93e2-ec508e0cf013 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:39.317616 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-22d8f8b6-6226-4cf7-93e2-ec508e0cf013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:39.326095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 603/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:39.614635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f0242-653b-4b32-a0bd-5d9649c294f7 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:39.616415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-035f0242-653b-4b32-a0bd-5d9649c294f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:39.680688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:39.695934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035f0242-653b-4b32-a0bd-5d9649c294f7 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:39.699500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb41a22-14e6-42a6-8372-0bf47d4e4d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:39.700708 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41a22-14e6-42a6-8372-0bf47d4e4d92 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:39.701003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41a22-14e6-42a6-8372-0bf47d4e4d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:39.701269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb41a22-14e6-42a6-8372-0bf47d4e4d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:39.710050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb41a22-14e6-42a6-8372-0bf47d4e4d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:39.711531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb41a22-14e6-42a6-8372-0bf47d4e4d92 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:39.721261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 604/1207] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:41.871354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5b755-b4c8-4304-9e50-81d475816cc2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:41.872021 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5b755-b4c8-4304-9e50-81d475816cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:41.872021 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5b755-b4c8-4304-9e50-81d475816cc2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:41.873287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 604/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:41.878702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fb171-a7ab-431b-9571-3087c0608244 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:41.879138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fb171-a7ab-431b-9571-3087c0608244 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:41.879312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fb171-a7ab-431b-9571-3087c0608244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:41.899717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7d8fb171-a7ab-431b-9571-3087c0608244 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:41.900926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7d8fb171-a7ab-431b-9571-3087c0608244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:41.909868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 605/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:42.201123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5b755-b4c8-4304-9e50-81d475816cc2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:42.212008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5b755-b4c8-4304-9e50-81d475816cc2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:42.215580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c124090-1b67-40f3-a9dd-0a46f7bcd1e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:42.216260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124090-1b67-40f3-a9dd-0a46f7bcd1e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:42.216550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124090-1b67-40f3-a9dd-0a46f7bcd1e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:42.216832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c124090-1b67-40f3-a9dd-0a46f7bcd1e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:42.225580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c124090-1b67-40f3-a9dd-0a46f7bcd1e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:42.227008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c124090-1b67-40f3-a9dd-0a46f7bcd1e8 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:42.236482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 605/1210] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:46.236347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05932b07-8ac8-43f2-91b0-b37ed78423f7 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:46.236840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05932b07-8ac8-43f2-91b0-b37ed78423f7 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:46.236940 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05932b07-8ac8-43f2-91b0-b37ed78423f7 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:46.238589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 606/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:46.243994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a995472-1fa4-42d8-8133-be8b92cd77e1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:46.244445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a995472-1fa4-42d8-8133-be8b92cd77e1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:46.244546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a995472-1fa4-42d8-8133-be8b92cd77e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:46.264726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6a995472-1fa4-42d8-8133-be8b92cd77e1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:46.265920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6a995472-1fa4-42d8-8133-be8b92cd77e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:46.273967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 606/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:46.574075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05932b07-8ac8-43f2-91b0-b37ed78423f7 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:46.576489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-05932b07-8ac8-43f2-91b0-b37ed78423f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:46.654020 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05932b07-8ac8-43f2-91b0-b37ed78423f7 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:46.657559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdc8d5-a294-4c30-b25a-712dd2e88bad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:46.658165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdc8d5-a294-4c30-b25a-712dd2e88bad service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:46.658448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdc8d5-a294-4c30-b25a-712dd2e88bad service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:46.658723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdc8d5-a294-4c30-b25a-712dd2e88bad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:46.667434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdc8d5-a294-4c30-b25a-712dd2e88bad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:46.668931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdc8d5-a294-4c30-b25a-712dd2e88bad service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:46.678045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 607/1213] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:48.830119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088adb30-db62-4363-9517-f1c54079e54c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:48.830698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088adb30-db62-4363-9517-f1c54079e54c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:48.830698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088adb30-db62-4363-9517-f1c54079e54c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:48.832069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 607/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 24 13:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:10:48.837400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f234a-bd50-4431-b90b-eba245d41da8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:48.837843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f234a-bd50-4431-b90b-eba245d41da8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:48.837944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f234a-bd50-4431-b90b-eba245d41da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:48.857403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-067f234a-bd50-4431-b90b-eba245d41da8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:10:48.858614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-067f234a-bd50-4431-b90b-eba245d41da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:48.866383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 608/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 24 13:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:10:49.171235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088adb30-db62-4363-9517-f1c54079e54c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:10:49.172913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-088adb30-db62-4363-9517-f1c54079e54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:10:49.185830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088adb30-db62-4363-9517-f1c54079e54c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:10:49.189317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0b425-951f-4e03-a792-51bed302a7e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:10:49.189912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0b425-951f-4e03-a792-51bed302a7e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:10:49.190197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0b425-951f-4e03-a792-51bed302a7e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:10:49.190467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0b425-951f-4e03-a792-51bed302a7e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:10:49.199119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0b425-951f-4e03-a792-51bed302a7e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:10:49.201066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0b425-951f-4e03-a792-51bed302a7e4 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:10:49.210378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 608/1216] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:10:58.852553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=50 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:10:59.206143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=50 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:13.650134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39b37-2f86-4c35-85a4-79cbeaab7102 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.651433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39b37-2f86-4c35-85a4-79cbeaab7102 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.652075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf39b37-2f86-4c35-85a4-79cbeaab7102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.688293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-2cf39b37-2f86-4c35-85a4-79cbeaab7102 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:13.689911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2cf39b37-2f86-4c35-85a4-79cbeaab7102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:13.701567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 609/1217] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:12:13.707776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aaf25-770d-42d5-b23b-7f3ee3a4b0cb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.708907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aaf25-770d-42d5-b23b-7f3ee3a4b0cb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.709462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aaf25-770d-42d5-b23b-7f3ee3a4b0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.735148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2e7c2-efe5-4393-a3e4-3ac320063134 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.735510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2e7c2-efe5-4393-a3e4-3ac320063134 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.735702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2e7c2-efe5-4393-a3e4-3ac320063134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.740142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-eb7aaf25-770d-42d5-b23b-7f3ee3a4b0cb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:13.741623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-eb7aaf25-770d-42d5-b23b-7f3ee3a4b0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:13.756053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 609/1218] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:13.770043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-39c2e7c2-efe5-4393-a3e4-3ac320063134 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:13.770043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-39c2e7c2-efe5-4393-a3e4-3ac320063134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:13.773415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094a4c1-2c67-49fa-a34e-d24861199f0b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.773982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094a4c1-2c67-49fa-a34e-d24861199f0b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.774472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094a4c1-2c67-49fa-a34e-d24861199f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.794637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 610/1219] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:13.796017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc186fb-fd39-41ee-8a10-d2109755c0fa None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.797488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc186fb-fd39-41ee-8a10-d2109755c0fa None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.797488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc186fb-fd39-41ee-8a10-d2109755c0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.811557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4094a4c1-2c67-49fa-a34e-d24861199f0b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:13.814520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4094a4c1-2c67-49fa-a34e-d24861199f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:13.843481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 610/1220] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:13.846073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094a4c1-2c67-49fa-a34e-d24861199f0b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:13.863611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0cc186fb-fd39-41ee-8a10-d2109755c0fa None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:13.867293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0cc186fb-fd39-41ee-8a10-d2109755c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:13.881357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094a4c1-2c67-49fa-a34e-d24861199f0b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:13.898359 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 611/1221] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:13.899748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:13.901343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.902436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af16ae-5636-47fd-9d07-7d38889c711e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.902955 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.903643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.904535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:13.905049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af16ae-5636-47fd-9d07-7d38889c711e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.905548 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af16ae-5636-47fd-9d07-7d38889c711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.906971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:13.915102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:13.947741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 611/1222] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:13.949787 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:13.960555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-84af16ae-5636-47fd-9d07-7d38889c711e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:13.961776 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-84af16ae-5636-47fd-9d07-7d38889c711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:13.972070 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7328d-b9c2-42a5-b87e-458423723ef0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:13.979801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 612/1223] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:13.985236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402278f-57f3-4e6f-82e0-cb18bf129ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:13.985864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402278f-57f3-4e6f-82e0-cb18bf129ade admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.986153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402278f-57f3-4e6f-82e0-cb18bf129ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.987886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79b4d7-b8e7-431c-adee-bbc8c71e0d7d None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:13.988304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79b4d7-b8e7-431c-adee-bbc8c71e0d7d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:13.988590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79b4d7-b8e7-431c-adee-bbc8c71e0d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.991470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402278f-57f3-4e6f-82e0-cb18bf129ade admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:13.992139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402278f-57f3-4e6f-82e0-cb18bf129ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:13.992455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402278f-57f3-4e6f-82e0-cb18bf129ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:13.994289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402278f-57f3-4e6f-82e0-cb18bf129ade admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.021848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 612/1224] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:14.023916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d6898-5118-4157-a99c-634bc9b2a640 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.024216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d6898-5118-4157-a99c-634bc9b2a640 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.024503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d6898-5118-4157-a99c-634bc9b2a640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.029641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7c79b4d7-b8e7-431c-adee-bbc8c71e0d7d None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.030855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c79b4d7-b8e7-431c-adee-bbc8c71e0d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.039913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 613/1225] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.040806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79b4d7-b8e7-431c-adee-bbc8c71e0d7d None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.048579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8f8d6898-5118-4157-a99c-634bc9b2a640 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.049890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8f8d6898-5118-4157-a99c-634bc9b2a640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.055636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c79b4d7-b8e7-431c-adee-bbc8c71e0d7d None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.060872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e5ea5-2a5b-4ba3-9fd9-7ce21484a715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.061486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5ea5-2a5b-4ba3-9fd9-7ce21484a715 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.061769 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5ea5-2a5b-4ba3-9fd9-7ce21484a715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.062044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e5ea5-2a5b-4ba3-9fd9-7ce21484a715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.066473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 613/1226] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.066636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e5ea5-2a5b-4ba3-9fd9-7ce21484a715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.066747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9c776-189e-4898-93fe-2d455a7c410c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.066747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9c776-189e-4898-93fe-2d455a7c410c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.066747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9c776-189e-4898-93fe-2d455a7c410c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.066975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e5ea5-2a5b-4ba3-9fd9-7ce21484a715 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.100289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-eec9c776-189e-4898-93fe-2d455a7c410c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.101818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-eec9c776-189e-4898-93fe-2d455a7c410c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.113250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 614/1227] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.114766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82cbd2-4888-42bd-ab96-05afdb807027 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.115156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82cbd2-4888-42bd-ab96-05afdb807027 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.115501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82cbd2-4888-42bd-ab96-05afdb807027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.122223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 614/1228] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/projects => generated 375 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.123445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e5ea5-2a5b-4ba3-9fd9-7ce21484a715 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.144915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8c82cbd2-4888-42bd-ab96-05afdb807027 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.146124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8c82cbd2-4888-42bd-ab96-05afdb807027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.194020 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e5ea5-2a5b-4ba3-9fd9-7ce21484a715 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.198730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cc01f-3155-47fa-9412-667012b8715e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.199378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc01f-3155-47fa-9412-667012b8715e admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.199659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc01f-3155-47fa-9412-667012b8715e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.199930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc01f-3155-47fa-9412-667012b8715e admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.200561 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cc01f-3155-47fa-9412-667012b8715e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.202445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cc01f-3155-47fa-9412-667012b8715e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.209137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 615/1229] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.210120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82cbd2-4888-42bd-ab96-05afdb807027 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.219341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82cbd2-4888-42bd-ab96-05afdb807027 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.224746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cd15d-a945-4fdb-b4ba-033e5e5b8bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.225335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cd15d-a945-4fdb-b4ba-033e5e5b8bab admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.225648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cd15d-a945-4fdb-b4ba-033e5e5b8bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.225967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cd15d-a945-4fdb-b4ba-033e5e5b8bab admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.226775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cd15d-a945-4fdb-b4ba-033e5e5b8bab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:14.227101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cd15d-a945-4fdb-b4ba-033e5e5b8bab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.228686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cd15d-a945-4fdb-b4ba-033e5e5b8bab admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.231528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 615/1230] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.232748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd1df-f1fc-4b02-b957-7fe2d6f7cd8f None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.233035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd1df-f1fc-4b02-b957-7fe2d6f7cd8f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.233309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afd1df-f1fc-4b02-b957-7fe2d6f7cd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.240904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 616/1231] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:14.242210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895930a-40fe-4b8c-a69b-0c758c128091 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.242494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895930a-40fe-4b8c-a69b-0c758c128091 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.242771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895930a-40fe-4b8c-a69b-0c758c128091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.260326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-38afd1df-f1fc-4b02-b957-7fe2d6f7cd8f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.261546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-38afd1df-f1fc-4b02-b957-7fe2d6f7cd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.265227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8895930a-40fe-4b8c-a69b-0c758c128091 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.267283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8895930a-40fe-4b8c-a69b-0c758c128091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.328549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 616/1232] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.329800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3a59-21d8-44bb-aae9-2e0e9895f9a5 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.330083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3a59-21d8-44bb-aae9-2e0e9895f9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.330358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f3a59-21d8-44bb-aae9-2e0e9895f9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.333094 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 617/1233] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.334141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895930a-40fe-4b8c-a69b-0c758c128091 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.345890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895930a-40fe-4b8c-a69b-0c758c128091 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.351237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e18f3a59-21d8-44bb-aae9-2e0e9895f9a5 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.351926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5349c-e2e9-4b56-88c6-b9a958b18cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.352429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e18f3a59-21d8-44bb-aae9-2e0e9895f9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.352549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5349c-e2e9-4b56-88c6-b9a958b18cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.352807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5349c-e2e9-4b56-88c6-b9a958b18cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.353085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5349c-e2e9-4b56-88c6-b9a958b18cfc admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.353725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5349c-e2e9-4b56-88c6-b9a958b18cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.355682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5349c-e2e9-4b56-88c6-b9a958b18cfc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.361209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 617/1234] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.362577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e52ed-5605-4116-9838-a72f78fde1df None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.362868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e52ed-5605-4116-9838-a72f78fde1df None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.363184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e52ed-5605-4116-9838-a72f78fde1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.381137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 618/1235] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/users => generated 288 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.381137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5349c-e2e9-4b56-88c6-b9a958b18cfc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.385239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a57e52ed-5605-4116-9838-a72f78fde1df None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.386441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a57e52ed-5605-4116-9838-a72f78fde1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.388541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5349c-e2e9-4b56-88c6-b9a958b18cfc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.393922 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.394525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.394815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.395131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.395248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 618/1236] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.395861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:14.396237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.397885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.402902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e52ed-5605-4116-9838-a72f78fde1df None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.417672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 619/1237] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:14.417672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.417972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e52ed-5605-4116-9838-a72f78fde1df None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.417972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.418155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:14.418262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.418262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.418643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.419255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.420381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING py.warnings [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:12:14.420381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: warnings.warn(msg) Oct 24 13:12:14.420381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: Oct 24 13:12:14.421609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.422004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e112f-d773-4388-a737-9972430ebca2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.428139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c89df8-b691-4f83-ab93-dd34b2cdf615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.428967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89df8-b691-4f83-ab93-dd34b2cdf615 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.429565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89df8-b691-4f83-ab93-dd34b2cdf615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.429849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89df8-b691-4f83-ab93-dd34b2cdf615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.430513 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c89df8-b691-4f83-ab93-dd34b2cdf615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.431385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:14.431599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-65c89df8-b691-4f83-ab93-dd34b2cdf615 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:12:14.431599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 13:12:14.431599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 13:12:14.432924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c89df8-b691-4f83-ab93-dd34b2cdf615 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.451748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 619/1238] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/users => generated 291 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.451748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.460094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1968fbc8-65ed-4330-bc81-ea9ce8d441fe admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.467071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 620/1239] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.467258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.467392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85fbe1-a113-4440-85c3-277cfb46dd54 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.467494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.467582 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85fbe1-a113-4440-85c3-277cfb46dd54 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.467683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.467779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c85fbe1-a113-4440-85c3-277cfb46dd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.467875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.467875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:14.467875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.468155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING py.warnings [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:12:14.468155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: warnings.warn(msg) Oct 24 13:12:14.468155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: Oct 24 13:12:14.469070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.482187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 620/1240] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:14.482986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.488850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9c85fbe1-a113-4440-85c3-277cfb46dd54 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.490556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9c85fbe1-a113-4440-85c3-277cfb46dd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.494263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595738f6-a02b-4f4a-aa74-9c246f88c3ad admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.558416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a9451-d2e6-407b-b9f8-0617c63fffac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.559267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a9451-d2e6-407b-b9f8-0617c63fffac admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.559693 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a9451-d2e6-407b-b9f8-0617c63fffac admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.560100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a9451-d2e6-407b-b9f8-0617c63fffac admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.560859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 621/1241] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.561872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85fbe1-a113-4440-85c3-277cfb46dd54 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.563367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a9451-d2e6-407b-b9f8-0617c63fffac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:14.563918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a9451-d2e6-407b-b9f8-0617c63fffac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.567270 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a9451-d2e6-407b-b9f8-0617c63fffac admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.572903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c85fbe1-a113-4440-85c3-277cfb46dd54 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.578470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8305dc-ca0f-4f34-8f0e-5eeb1c1c3e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.578893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8305dc-ca0f-4f34-8f0e-5eeb1c1c3e01 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.579364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8305dc-ca0f-4f34-8f0e-5eeb1c1c3e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.579703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8305dc-ca0f-4f34-8f0e-5eeb1c1c3e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.580579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8305dc-ca0f-4f34-8f0e-5eeb1c1c3e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.582276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 621/1242] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:14.582752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8305dc-ca0f-4f34-8f0e-5eeb1c1c3e01 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.583622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff19d16-121c-4392-8df9-0ac3d44c7c7b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.583910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff19d16-121c-4392-8df9-0ac3d44c7c7b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.584186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff19d16-121c-4392-8df9-0ac3d44c7c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.598864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 622/1243] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:14.599698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8305dc-ca0f-4f34-8f0e-5eeb1c1c3e01 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.610171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8305dc-ca0f-4f34-8f0e-5eeb1c1c3e01 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.612392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5ff19d16-121c-4392-8df9-0ac3d44c7c7b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:14.613692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5ff19d16-121c-4392-8df9-0ac3d44c7c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:14.615179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e735a5b-7b7a-474b-860e-b361f4b2c488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.615785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e735a5b-7b7a-474b-860e-b361f4b2c488 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.616069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e735a5b-7b7a-474b-860e-b361f4b2c488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.616344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e735a5b-7b7a-474b-860e-b361f4b2c488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.616975 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e735a5b-7b7a-474b-860e-b361f4b2c488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.618923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e735a5b-7b7a-474b-860e-b361f4b2c488 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.621666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 622/1244] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:14.622572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff19d16-121c-4392-8df9-0ac3d44c7c7b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.632546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff19d16-121c-4392-8df9-0ac3d44c7c7b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.637402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3dd23-408e-41d4-88dc-73a5690d9a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.638012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3dd23-408e-41d4-88dc-73a5690d9a8f admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.638344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3dd23-408e-41d4-88dc-73a5690d9a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.638573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3dd23-408e-41d4-88dc-73a5690d9a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.639261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3dd23-408e-41d4-88dc-73a5690d9a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.640910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3dd23-408e-41d4-88dc-73a5690d9a8f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.654323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 623/1245] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:14.655182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3dd23-408e-41d4-88dc-73a5690d9a8f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.664551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 623/1246] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/projects => generated 393 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.665389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e735a5b-7b7a-474b-860e-b361f4b2c488 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.666834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3dd23-408e-41d4-88dc-73a5690d9a8f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.727988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e735a5b-7b7a-474b-860e-b361f4b2c488 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.729185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0bc12-3c7e-4721-8054-6c2fc8d53e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.729797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0bc12-3c7e-4721-8054-6c2fc8d53e9b admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.730081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0bc12-3c7e-4721-8054-6c2fc8d53e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.730356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0bc12-3c7e-4721-8054-6c2fc8d53e9b admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.730976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0bc12-3c7e-4721-8054-6c2fc8d53e9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.732664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b2e88-8975-4c37-9add-99cf745dec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.732935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0bc12-3c7e-4721-8054-6c2fc8d53e9b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.733268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b2e88-8975-4c37-9add-99cf745dec08 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.733557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b2e88-8975-4c37-9add-99cf745dec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.733832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b2e88-8975-4c37-9add-99cf745dec08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.735324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b2e88-8975-4c37-9add-99cf745dec08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.736956 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b2e88-8975-4c37-9add-99cf745dec08 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.757223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 624/1247] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.758147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0bc12-3c7e-4721-8054-6c2fc8d53e9b admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.768573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0bc12-3c7e-4721-8054-6c2fc8d53e9b admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.770979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 624/1248] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.772015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b2e88-8975-4c37-9add-99cf745dec08 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.815902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:14.821720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:14.839290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.839914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.840330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.840616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.841287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:14.841594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.843251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.845347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b2e88-8975-4c37-9add-99cf745dec08 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.850196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c1c29-024a-46fa-a564-1df2bc1c6e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.856112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c1c29-024a-46fa-a564-1df2bc1c6e10 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.856112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c1c29-024a-46fa-a564-1df2bc1c6e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.856112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c1c29-024a-46fa-a564-1df2bc1c6e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.856112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c1c29-024a-46fa-a564-1df2bc1c6e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.856112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c1c29-024a-46fa-a564-1df2bc1c6e10 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.857530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 625/1249] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:14.858682 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.871357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7d357-be68-446d-86a3-d5381126e9af admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.876930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7edcc-cc59-4527-9fa8-5d2e881e8dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.877544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7edcc-cc59-4527-9fa8-5d2e881e8dee admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.877820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7edcc-cc59-4527-9fa8-5d2e881e8dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.878143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7edcc-cc59-4527-9fa8-5d2e881e8dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.878855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7edcc-cc59-4527-9fa8-5d2e881e8dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.880517 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7edcc-cc59-4527-9fa8-5d2e881e8dee admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.889250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 625/1250] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:14.890570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c1c29-024a-46fa-a564-1df2bc1c6e10 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.891888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 626/1251] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:14.892679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7edcc-cc59-4527-9fa8-5d2e881e8dee admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.903734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7edcc-cc59-4527-9fa8-5d2e881e8dee admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.952558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c1c29-024a-46fa-a564-1df2bc1c6e10 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.957431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.958050 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.958319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.958590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.959257 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:14.959561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.960090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6dfd-f98d-4f0c-ae82-79e261a1cb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.960703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6dfd-f98d-4f0c-ae82-79e261a1cb20 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.960985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6dfd-f98d-4f0c-ae82-79e261a1cb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.961140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.961259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6dfd-f98d-4f0c-ae82-79e261a1cb20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.961886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6dfd-f98d-4f0c-ae82-79e261a1cb20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.963500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6dfd-f98d-4f0c-ae82-79e261a1cb20 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.973176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 626/1252] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:14.976969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 627/1253] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:14.977126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.977223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6dfd-f98d-4f0c-ae82-79e261a1cb20 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:14.989909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab0aef-eb8a-48d1-a19a-a91eeb76e0d7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.990075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6dfd-f98d-4f0c-ae82-79e261a1cb20 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:14.994279 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e94b3-5974-4637-aa5c-31ecff5dedd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.994632 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c16f8-b3f2-41cd-a8b0-59b6a7f49e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:14.994868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e94b3-5974-4637-aa5c-31ecff5dedd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.995189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e94b3-5974-4637-aa5c-31ecff5dedd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.995311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c16f8-b3f2-41cd-a8b0-59b6a7f49e08 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:14.995467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e94b3-5974-4637-aa5c-31ecff5dedd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.995577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c16f8-b3f2-41cd-a8b0-59b6a7f49e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:14.995787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1c16f8-b3f2-41cd-a8b0-59b6a7f49e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:14.996084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e94b3-5974-4637-aa5c-31ecff5dedd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.996422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c16f8-b3f2-41cd-a8b0-59b6a7f49e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:14.998109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e94b3-5974-4637-aa5c-31ecff5dedd1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:14.998669 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1c16f8-b3f2-41cd-a8b0-59b6a7f49e08 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.011666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 628/1254] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.012724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c16f8-b3f2-41cd-a8b0-59b6a7f49e08 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.020455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 627/1255] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:14 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:15.021287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e94b3-5974-4637-aa5c-31ecff5dedd1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.025819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1c16f8-b3f2-41cd-a8b0-59b6a7f49e08 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.030652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e69b-6699-4238-87cb-507c2854ef42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.031634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e94b3-5974-4637-aa5c-31ecff5dedd1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.031879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e69b-6699-4238-87cb-507c2854ef42 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.032161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e69b-6699-4238-87cb-507c2854ef42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.032433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e69b-6699-4238-87cb-507c2854ef42 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.033085 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1e69b-6699-4238-87cb-507c2854ef42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.034978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1e69b-6699-4238-87cb-507c2854ef42 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.036302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123910f-d3d5-4498-bb32-e58b7290d0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.036894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123910f-d3d5-4498-bb32-e58b7290d0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.037181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123910f-d3d5-4498-bb32-e58b7290d0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.037459 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123910f-d3d5-4498-bb32-e58b7290d0f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.038083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123910f-d3d5-4498-bb32-e58b7290d0f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.040003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123910f-d3d5-4498-bb32-e58b7290d0f0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.057165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 629/1256] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/users => generated 303 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:15.058366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e69b-6699-4238-87cb-507c2854ef42 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.068501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e69b-6699-4238-87cb-507c2854ef42 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.072239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 628/1257] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:15.072838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123910f-d3d5-4498-bb32-e58b7290d0f0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.076660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30675174-2e7c-4e3c-8e33-34b4c72e9216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.077334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30675174-2e7c-4e3c-8e33-34b4c72e9216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.077617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30675174-2e7c-4e3c-8e33-34b4c72e9216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.077942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30675174-2e7c-4e3c-8e33-34b4c72e9216 admin admin] PATH_INFO: `/v3/projects/c837e1766e0141ea9b868b89efee5ba1/users/b3e637d319f449f38e4d6375ae5b22fc/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.092714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30675174-2e7c-4e3c-8e33-34b4c72e9216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c837e1766e0141ea9b868b89efee5ba1, user_id=b3e637d319f449f38e4d6375ae5b22fc, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.095938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30675174-2e7c-4e3c-8e33-34b4c72e9216 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.110758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 630/1258] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:15 2022] PUT /identity/v3/projects/c837e1766e0141ea9b868b89efee5ba1/users/b3e637d319f449f38e4d6375ae5b22fc/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:15.111613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30675174-2e7c-4e3c-8e33-34b4c72e9216 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.143608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123910f-d3d5-4498-bb32-e58b7290d0f0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.162770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:15.177935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30675174-2e7c-4e3c-8e33-34b4c72e9216 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.182537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08983a5-d8f0-421d-a705-52d3aec7fb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.183247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08983a5-d8f0-421d-a705-52d3aec7fb8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.183500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08983a5-d8f0-421d-a705-52d3aec7fb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.183781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08983a5-d8f0-421d-a705-52d3aec7fb8a admin admin] PATH_INFO: `/v3/projects/1db6666d568b466ea2fa1d3c450fd380/users/ecd03c36e31e42588a5b87d3d3303aa9/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.199036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08983a5-d8f0-421d-a705-52d3aec7fb8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1db6666d568b466ea2fa1d3c450fd380, user_id=ecd03c36e31e42588a5b87d3d3303aa9, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.201746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08983a5-d8f0-421d-a705-52d3aec7fb8a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.203826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e457800-4d46-4b40-bd16-17c66e01202a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.204281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:15.205385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e457800-4d46-4b40-bd16-17c66e01202a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.205758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e457800-4d46-4b40-bd16-17c66e01202a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.206104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e457800-4d46-4b40-bd16-17c66e01202a admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.207144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e457800-4d46-4b40-bd16-17c66e01202a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.209600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e457800-4d46-4b40-bd16-17c66e01202a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.216127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 631/1259] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:15 2022] PUT /identity/v3/projects/1db6666d568b466ea2fa1d3c450fd380/users/ecd03c36e31e42588a5b87d3d3303aa9/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:15.217035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08983a5-d8f0-421d-a705-52d3aec7fb8a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.232390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 629/1260] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:15.233220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e457800-4d46-4b40-bd16-17c66e01202a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.243239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e457800-4d46-4b40-bd16-17c66e01202a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.291333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08983a5-d8f0-421d-a705-52d3aec7fb8a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.296254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cfd4b-0da3-4f48-8c48-3adbcbc1de14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.297715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cfd4b-0da3-4f48-8c48-3adbcbc1de14 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.298012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cfd4b-0da3-4f48-8c48-3adbcbc1de14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.298295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cfd4b-0da3-4f48-8c48-3adbcbc1de14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.298930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cfd4b-0da3-4f48-8c48-3adbcbc1de14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.301055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cfd4b-0da3-4f48-8c48-3adbcbc1de14 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.301762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab918894-38b8-411d-93f8-4b53009b1572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.302373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab918894-38b8-411d-93f8-4b53009b1572 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.302655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab918894-38b8-411d-93f8-4b53009b1572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.302958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab918894-38b8-411d-93f8-4b53009b1572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.303630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab918894-38b8-411d-93f8-4b53009b1572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.305232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab918894-38b8-411d-93f8-4b53009b1572 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.315858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 630/1261] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.316661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab918894-38b8-411d-93f8-4b53009b1572 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.329572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab918894-38b8-411d-93f8-4b53009b1572 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.330348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 632/1262] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:15.332008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cfd4b-0da3-4f48-8c48-3adbcbc1de14 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.387582 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329548b-1a22-4548-8475-d8809bc98e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.388306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329548b-1a22-4548-8475-d8809bc98e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.388640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329548b-1a22-4548-8475-d8809bc98e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.388991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6329548b-1a22-4548-8475-d8809bc98e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.389703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329548b-1a22-4548-8475-d8809bc98e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.392381 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6329548b-1a22-4548-8475-d8809bc98e9a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.395793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cfd4b-0da3-4f48-8c48-3adbcbc1de14 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.400373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd96ebb-630b-4e7a-99b3-a0bc882c71ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.400973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96ebb-630b-4e7a-99b3-a0bc882c71ad admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.401252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96ebb-630b-4e7a-99b3-a0bc882c71ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.401519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd96ebb-630b-4e7a-99b3-a0bc882c71ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.402138 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 631/1263] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.402289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd96ebb-630b-4e7a-99b3-a0bc882c71ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.402913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329548b-1a22-4548-8475-d8809bc98e9a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.403777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd96ebb-630b-4e7a-99b3-a0bc882c71ad admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.412736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6329548b-1a22-4548-8475-d8809bc98e9a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.417789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 633/1264] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.417789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145128c-5ce3-4231-a38e-a46afc2444e9 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.417789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145128c-5ce3-4231-a38e-a46afc2444e9 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.417789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145128c-5ce3-4231-a38e-a46afc2444e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.420450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94794867-7dfb-4cf6-9831-3057d3d60a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.421125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94794867-7dfb-4cf6-9831-3057d3d60a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.421980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94794867-7dfb-4cf6-9831-3057d3d60a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.422282 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94794867-7dfb-4cf6-9831-3057d3d60a5e admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.422913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94794867-7dfb-4cf6-9831-3057d3d60a5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.424890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94794867-7dfb-4cf6-9831-3057d3d60a5e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.437261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4145128c-5ce3-4231-a38e-a46afc2444e9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:15.438493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4145128c-5ce3-4231-a38e-a46afc2444e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:15.446756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 634/1265] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:15.448329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d38530-9c4b-4d51-ab4f-3c5a1407f424 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.448616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d38530-9c4b-4d51-ab4f-3c5a1407f424 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.448895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d38530-9c4b-4d51-ab4f-3c5a1407f424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.450182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 632/1266] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/users => generated 282 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:15.451343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94794867-7dfb-4cf6-9831-3057d3d60a5e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.462210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94794867-7dfb-4cf6-9831-3057d3d60a5e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.466833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec0230-2044-4c2a-99b4-a3458dd5b31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.467479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec0230-2044-4c2a-99b4-a3458dd5b31d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.467765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec0230-2044-4c2a-99b4-a3458dd5b31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.468043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec0230-2044-4c2a-99b4-a3458dd5b31d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.469402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec0230-2044-4c2a-99b4-a3458dd5b31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.471230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec0230-2044-4c2a-99b4-a3458dd5b31d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.479087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-85d38530-9c4b-4d51-ab4f-3c5a1407f424 None None] MFA Rules not processed for user `ecd03c36e31e42588a5b87d3d3303aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:15.480296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-85d38530-9c4b-4d51-ab4f-3c5a1407f424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:15.481916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 633/1267] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.482490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec0230-2044-4c2a-99b4-a3458dd5b31d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.493028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec0230-2044-4c2a-99b4-a3458dd5b31d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.497411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86788ca5-180f-423c-b99b-29268afbd10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.498000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86788ca5-180f-423c-b99b-29268afbd10b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.498284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86788ca5-180f-423c-b99b-29268afbd10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.498564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86788ca5-180f-423c-b99b-29268afbd10b admin admin] PATH_INFO: `/v3/projects/69beb82dd21b447d8a784e0570c4323c/users/c5619c8183e04e1d9073f1b40e057967/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.511196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:15.516030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86788ca5-180f-423c-b99b-29268afbd10b admin admin] RBAC: Authorizing `identity:create_grant(project_id=69beb82dd21b447d8a784e0570c4323c, user_id=c5619c8183e04e1d9073f1b40e057967, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.519291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86788ca5-180f-423c-b99b-29268afbd10b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.537069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 634/1268] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:15 2022] PUT /identity/v3/projects/69beb82dd21b447d8a784e0570c4323c/users/c5619c8183e04e1d9073f1b40e057967/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:15.541272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86788ca5-180f-423c-b99b-29268afbd10b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.555117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:15.563690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 635/1269] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:15.564551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d38530-9c4b-4d51-ab4f-3c5a1407f424 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.574521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d38530-9c4b-4d51-ab4f-3c5a1407f424 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.602392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86788ca5-180f-423c-b99b-29268afbd10b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.608193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d91ee3-22d4-454b-b0cb-a4880a2d7405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.608809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d91ee3-22d4-454b-b0cb-a4880a2d7405 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.609117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d91ee3-22d4-454b-b0cb-a4880a2d7405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.609394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d91ee3-22d4-454b-b0cb-a4880a2d7405 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.610729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d91ee3-22d4-454b-b0cb-a4880a2d7405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.612487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d91ee3-22d4-454b-b0cb-a4880a2d7405 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.641361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 635/1270] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:15.642755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09781fd0-ffb6-40fb-885a-441e7825ded4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.642932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d91ee3-22d4-454b-b0cb-a4880a2d7405 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.643424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09781fd0-ffb6-40fb-885a-441e7825ded4 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.643705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09781fd0-ffb6-40fb-885a-441e7825ded4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.643983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09781fd0-ffb6-40fb-885a-441e7825ded4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.644730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09781fd0-ffb6-40fb-885a-441e7825ded4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.646548 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09781fd0-ffb6-40fb-885a-441e7825ded4 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.652840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d91ee3-22d4-454b-b0cb-a4880a2d7405 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.660189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 636/1271] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.660189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09781fd0-ffb6-40fb-885a-441e7825ded4 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.661449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116ea2c-671f-42b2-af94-d782dc2e5ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.662085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116ea2c-671f-42b2-af94-d782dc2e5ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.662373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116ea2c-671f-42b2-af94-d782dc2e5ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.662672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116ea2c-671f-42b2-af94-d782dc2e5ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.663362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116ea2c-671f-42b2-af94-d782dc2e5ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.665041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116ea2c-671f-42b2-af94-d782dc2e5ccb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.667481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09781fd0-ffb6-40fb-885a-441e7825ded4 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.672237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff375370-3e37-4cfd-b8ae-cb5951762ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.672831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff375370-3e37-4cfd-b8ae-cb5951762ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.673119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff375370-3e37-4cfd-b8ae-cb5951762ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.673398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff375370-3e37-4cfd-b8ae-cb5951762ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.674059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff375370-3e37-4cfd-b8ae-cb5951762ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.675646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff375370-3e37-4cfd-b8ae-cb5951762ddf admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.676039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 636/1272] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.676843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116ea2c-671f-42b2-af94-d782dc2e5ccb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.686382 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 637/1273] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.687075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff375370-3e37-4cfd-b8ae-cb5951762ddf admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.699960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff375370-3e37-4cfd-b8ae-cb5951762ddf admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.705235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9d5b6-e570-433b-8905-e082131888c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.705932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9d5b6-e570-433b-8905-e082131888c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.706208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9d5b6-e570-433b-8905-e082131888c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.706498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9d5b6-e570-433b-8905-e082131888c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.707165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9d5b6-e570-433b-8905-e082131888c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.709147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9d5b6-e570-433b-8905-e082131888c0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.719577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 638/1274] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.720787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0ee6-3e4b-45ee-8953-d2a0d0b31649 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.721109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0ee6-3e4b-45ee-8953-d2a0d0b31649 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.721390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be0ee6-3e4b-45ee-8953-d2a0d0b31649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.737562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116ea2c-671f-42b2-af94-d782dc2e5ccb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.740179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-b5be0ee6-3e4b-45ee-8953-d2a0d0b31649 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:15.741192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34a54-3b53-4b77-88c9-d74a8497fede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.741572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b5be0ee6-3e4b-45ee-8953-d2a0d0b31649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:15.741795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34a54-3b53-4b77-88c9-d74a8497fede service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.742074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34a54-3b53-4b77-88c9-d74a8497fede service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.742349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34a54-3b53-4b77-88c9-d74a8497fede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.750671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 639/1275] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:15.751429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be0ee6-3e4b-45ee-8953-d2a0d0b31649 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.753073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb34a54-3b53-4b77-88c9-d74a8497fede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.754501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb34a54-3b53-4b77-88c9-d74a8497fede service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.761002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be0ee6-3e4b-45ee-8953-d2a0d0b31649 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.766411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491ee50-0987-4952-8050-6a68ef1df519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.766722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 637/1276] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:15.767542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ee50-0987-4952-8050-6a68ef1df519 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.767671 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34a54-3b53-4b77-88c9-d74a8497fede service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.767847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ee50-0987-4952-8050-6a68ef1df519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.768201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ee50-0987-4952-8050-6a68ef1df519 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.768838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491ee50-0987-4952-8050-6a68ef1df519 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.770952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491ee50-0987-4952-8050-6a68ef1df519 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.776794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34a54-3b53-4b77-88c9-d74a8497fede service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.782067 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fffd4-e6c5-4f83-a195-d4f8c91fb3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.782725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fffd4-e6c5-4f83-a195-d4f8c91fb3e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.783019 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fffd4-e6c5-4f83-a195-d4f8c91fb3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.783327 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fffd4-e6c5-4f83-a195-d4f8c91fb3e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.784029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fffd4-e6c5-4f83-a195-d4f8c91fb3e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.785709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fffd4-e6c5-4f83-a195-d4f8c91fb3e7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.801127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 638/1277] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:15.804353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 640/1278] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:15 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:15.804476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fffd4-e6c5-4f83-a195-d4f8c91fb3e7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.804589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491ee50-0987-4952-8050-6a68ef1df519 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.815902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fffd4-e6c5-4f83-a195-d4f8c91fb3e7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.879527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491ee50-0987-4952-8050-6a68ef1df519 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:15.885005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81e368-69cc-4674-881b-0d16613a4576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.885768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e368-69cc-4674-881b-0d16613a4576 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.886364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e368-69cc-4674-881b-0d16613a4576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.886721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81e368-69cc-4674-881b-0d16613a4576 admin admin] PATH_INFO: `/v3/projects/07c5c1b5f3cb43e39ac7db457f1189aa/users/b3fd1f7c6b83402b8a7b7c7e5369acee/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.891355 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabb0bd-57a6-4b71-a92e-c2c2aab01d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:15.892040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabb0bd-57a6-4b71-a92e-c2c2aab01d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:15.892326 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabb0bd-57a6-4b71-a92e-c2c2aab01d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:15.892608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dabb0bd-57a6-4b71-a92e-c2c2aab01d6e admin admin] PATH_INFO: `/v3/projects/85d62dec32104e7b8c595953a479026b/users/d29ff25b062346f6bcf9910b1fa8db5c/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:15.896833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:15.907169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81e368-69cc-4674-881b-0d16613a4576 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c5c1b5f3cb43e39ac7db457f1189aa, user_id=b3fd1f7c6b83402b8a7b7c7e5369acee, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.909976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81e368-69cc-4674-881b-0d16613a4576 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.915161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabb0bd-57a6-4b71-a92e-c2c2aab01d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d62dec32104e7b8c595953a479026b, user_id=d29ff25b062346f6bcf9910b1fa8db5c, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:15.917934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dabb0bd-57a6-4b71-a92e-c2c2aab01d6e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:15.926287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 641/1279] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:15 2022] PUT /identity/v3/projects/07c5c1b5f3cb43e39ac7db457f1189aa/users/b3fd1f7c6b83402b8a7b7c7e5369acee/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:15.927460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81e368-69cc-4674-881b-0d16613a4576 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.934181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 639/1280] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:15 2022] PUT /identity/v3/projects/85d62dec32104e7b8c595953a479026b/users/d29ff25b062346f6bcf9910b1fa8db5c/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:15.935406 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabb0bd-57a6-4b71-a92e-c2c2aab01d6e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:15.968058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:16.013461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81e368-69cc-4674-881b-0d16613a4576 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.018624 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dabb0bd-57a6-4b71-a92e-c2c2aab01d6e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.018957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.019981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.020426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.020959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.022224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.024137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.024449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23d160-a6c6-4e3e-a33a-c6e97f6f21d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.025127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23d160-a6c6-4e3e-a33a-c6e97f6f21d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.025414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23d160-a6c6-4e3e-a33a-c6e97f6f21d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.025692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c23d160-a6c6-4e3e-a33a-c6e97f6f21d3 admin admin] PATH_INFO: `/v3/projects/b736b09b6ee2492590cbc23db48752f4/users/3e02f6f701324af6b53762569f1a9d21/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.036622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 642/1281] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:16.037950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.039799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:16.044486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23d160-a6c6-4e3e-a33a-c6e97f6f21d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b736b09b6ee2492590cbc23db48752f4, user_id=3e02f6f701324af6b53762569f1a9d21, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.048668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c23d160-a6c6-4e3e-a33a-c6e97f6f21d3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.068320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 640/1282] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:15 2022] PUT /identity/v3/projects/b736b09b6ee2492590cbc23db48752f4/users/3e02f6f701324af6b53762569f1a9d21/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:16.071970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23d160-a6c6-4e3e-a33a-c6e97f6f21d3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.104396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bbbf3e-1448-4ecc-b3ca-edf5273d4d48 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.154652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c23d160-a6c6-4e3e-a33a-c6e97f6f21d3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.159911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8efe7-e7f7-4fe8-ad3e-14c1a7b40b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.160533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8efe7-e7f7-4fe8-ad3e-14c1a7b40b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.160826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8efe7-e7f7-4fe8-ad3e-14c1a7b40b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.161160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8efe7-e7f7-4fe8-ad3e-14c1a7b40b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.161824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8efe7-e7f7-4fe8-ad3e-14c1a7b40b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.163749 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c8efe7-e7f7-4fe8-ad3e-14c1a7b40b0c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.175157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe566c-1d45-428b-8902-349c5d0e4a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.176036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe566c-1d45-428b-8902-349c5d0e4a49 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.176804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe566c-1d45-428b-8902-349c5d0e4a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.176920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 641/1283] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:16.177631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8efe7-e7f7-4fe8-ad3e-14c1a7b40b0c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.180044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe566c-1d45-428b-8902-349c5d0e4a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.189130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8efe7-e7f7-4fe8-ad3e-14c1a7b40b0c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.192647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe566c-1d45-428b-8902-349c5d0e4a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.194186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe566c-1d45-428b-8902-349c5d0e4a49 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.194307 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecb2428-b1a7-4f61-a10d-c7271d8248dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.194954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2428-b1a7-4f61-a10d-c7271d8248dd admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.195303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2428-b1a7-4f61-a10d-c7271d8248dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.195588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb2428-b1a7-4f61-a10d-c7271d8248dd admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.196440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb2428-b1a7-4f61-a10d-c7271d8248dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.198347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecb2428-b1a7-4f61-a10d-c7271d8248dd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.206870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 643/1284] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:16.208261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da8fcd-6247-45c0-b830-61bbc586d9c4 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.208554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da8fcd-6247-45c0-b830-61bbc586d9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.208845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da8fcd-6247-45c0-b830-61bbc586d9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.232529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 642/1285] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/users => generated 296 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:16.232529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2afa54-2fff-4370-a40f-8d114f2e5a74 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.232529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2afa54-2fff-4370-a40f-8d114f2e5a74 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.232529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2afa54-2fff-4370-a40f-8d114f2e5a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.247604 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-92da8fcd-6247-45c0-b830-61bbc586d9c4 None None] MFA Rules not processed for user `b3fd1f7c6b83402b8a7b7c7e5369acee`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:16.248723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-92da8fcd-6247-45c0-b830-61bbc586d9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:16.267833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db2afa54-2fff-4370-a40f-8d114f2e5a74 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:16.269018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db2afa54-2fff-4370-a40f-8d114f2e5a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:16.278178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 643/1286] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:16.278756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2afa54-2fff-4370-a40f-8d114f2e5a74 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.290196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2afa54-2fff-4370-a40f-8d114f2e5a74 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.295479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f09ab2-6bfb-49e9-b7ad-6e505d04d8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.296965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f09ab2-6bfb-49e9-b7ad-6e505d04d8b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.297376 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f09ab2-6bfb-49e9-b7ad-6e505d04d8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.297754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f09ab2-6bfb-49e9-b7ad-6e505d04d8b5 admin admin] PATH_INFO: `/v3/projects/d05ad5008e7c411da8d8f5d4762e55b1/users/e5bcdd36246d4731a7e2f43e1bd63a65/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.313180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f09ab2-6bfb-49e9-b7ad-6e505d04d8b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05ad5008e7c411da8d8f5d4762e55b1, user_id=e5bcdd36246d4731a7e2f43e1bd63a65, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.315918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f09ab2-6bfb-49e9-b7ad-6e505d04d8b5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.333937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 644/1287] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:16 2022] PUT /identity/v3/projects/d05ad5008e7c411da8d8f5d4762e55b1/users/e5bcdd36246d4731a7e2f43e1bd63a65/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:16.335411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78b2ff-89ff-40fa-9502-d7490a625cb1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.336025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 644/1288] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:16.337100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da8fcd-6247-45c0-b830-61bbc586d9c4 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.345507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78b2ff-89ff-40fa-9502-d7490a625cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.346038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78b2ff-89ff-40fa-9502-d7490a625cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.352481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da8fcd-6247-45c0-b830-61bbc586d9c4 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.361808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:16.368877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ab78b2ff-89ff-40fa-9502-d7490a625cb1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:16.370127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ab78b2ff-89ff-40fa-9502-d7490a625cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:16.382486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:16.421779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e66eb-3e1f-4045-8a49-201d970f812c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.422622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e66eb-3e1f-4045-8a49-201d970f812c admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.423014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e66eb-3e1f-4045-8a49-201d970f812c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.423449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e66eb-3e1f-4045-8a49-201d970f812c admin admin] PATH_INFO: `/v3/projects/54de867d20774b7c9d90e8f06795a918/users/b8c8e7a6f6a9447a8c24dad629dff6a6/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.443797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e66eb-3e1f-4045-8a49-201d970f812c admin admin] RBAC: Authorizing `identity:create_grant(project_id=54de867d20774b7c9d90e8f06795a918, user_id=b8c8e7a6f6a9447a8c24dad629dff6a6, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.446603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 645/1289] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:16.447560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78b2ff-89ff-40fa-9502-d7490a625cb1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.450253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e66eb-3e1f-4045-8a49-201d970f812c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.459158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78b2ff-89ff-40fa-9502-d7490a625cb1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.464904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7dfe1-a176-430b-be31-63b8a643d920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.465535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7dfe1-a176-430b-be31-63b8a643d920 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.465867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7dfe1-a176-430b-be31-63b8a643d920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.466124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7dfe1-a176-430b-be31-63b8a643d920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.466720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7dfe1-a176-430b-be31-63b8a643d920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.479004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7dfe1-a176-430b-be31-63b8a643d920 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.479257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 645/1290] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:16 2022] PUT /identity/v3/projects/54de867d20774b7c9d90e8f06795a918/users/b8c8e7a6f6a9447a8c24dad629dff6a6/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:16.479257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e66eb-3e1f-4045-8a49-201d970f812c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.480191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 646/1291] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:16.485133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7dfe1-a176-430b-be31-63b8a643d920 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.495526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7dfe1-a176-430b-be31-63b8a643d920 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.545545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e66eb-3e1f-4045-8a49-201d970f812c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.551094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888ea02-4e5d-4584-9ade-3aa543d752bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.553634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888ea02-4e5d-4584-9ade-3aa543d752bd service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.553942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888ea02-4e5d-4584-9ade-3aa543d752bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.555044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888ea02-4e5d-4584-9ade-3aa543d752bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.566630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed2213c-a0ad-421f-8e04-742a98fe6eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.567273 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2213c-a0ad-421f-8e04-742a98fe6eec admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.567557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2213c-a0ad-421f-8e04-742a98fe6eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.567834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2213c-a0ad-421f-8e04-742a98fe6eec admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.568483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2213c-a0ad-421f-8e04-742a98fe6eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.570399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed2213c-a0ad-421f-8e04-742a98fe6eec admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.607084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 647/1292] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/projects => generated 403 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:16.608599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c2ecf-4eb4-4b11-b78b-c2ae3789be46 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.608983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c2ecf-4eb4-4b11-b78b-c2ae3789be46 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.609271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c2ecf-4eb4-4b11-b78b-c2ae3789be46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.639796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888ea02-4e5d-4584-9ade-3aa543d752bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.641336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888ea02-4e5d-4584-9ade-3aa543d752bd service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.646008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2d5c2ecf-4eb4-4b11-b78b-c2ae3789be46 None None] MFA Rules not processed for user `e5bcdd36246d4731a7e2f43e1bd63a65`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:16.647447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2d5c2ecf-4eb4-4b11-b78b-c2ae3789be46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:16.653357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 646/1293] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:16.656845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888ea02-4e5d-4584-9ade-3aa543d752bd service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.669081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888ea02-4e5d-4584-9ade-3aa543d752bd service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.722673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:16.748745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c1254-9181-4688-9d15-7469818ee58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.749396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c1254-9181-4688-9d15-7469818ee58e service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.749751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c1254-9181-4688-9d15-7469818ee58e service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.750037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c1254-9181-4688-9d15-7469818ee58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.784328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 648/1294] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:16.785954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e09f5-ae2d-4798-88b8-219fbda57fd7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.786247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e09f5-ae2d-4798-88b8-219fbda57fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.786522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e09f5-ae2d-4798-88b8-219fbda57fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.824764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7e3e09f5-ae2d-4798-88b8-219fbda57fd7 None None] MFA Rules not processed for user `b8c8e7a6f6a9447a8c24dad629dff6a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:16.826252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7e3e09f5-ae2d-4798-88b8-219fbda57fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:16.842646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:16.860786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c1254-9181-4688-9d15-7469818ee58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.862410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c1254-9181-4688-9d15-7469818ee58e service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.878264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 647/1295] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:16.879242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c1254-9181-4688-9d15-7469818ee58e service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.891774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c1254-9181-4688-9d15-7469818ee58e service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.902800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55071b67-6c20-464c-b9a4-f8f659e3c29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.903522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55071b67-6c20-464c-b9a4-f8f659e3c29c admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.903814 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55071b67-6c20-464c-b9a4-f8f659e3c29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.904227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55071b67-6c20-464c-b9a4-f8f659e3c29c admin admin] PATH_INFO: `/v3/projects/770206629c1d4eb89e5ddf603e7708eb/users/edcc51c2df9e4c0aa71581b0c2f74f80/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.925615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 649/1296] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:16.927203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e09f5-ae2d-4798-88b8-219fbda57fd7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.927703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55071b67-6c20-464c-b9a4-f8f659e3c29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=770206629c1d4eb89e5ddf603e7708eb, user_id=edcc51c2df9e4c0aa71581b0c2f74f80, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.931986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55071b67-6c20-464c-b9a4-f8f659e3c29c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.940056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3e09f5-ae2d-4798-88b8-219fbda57fd7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:16.948555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08258111-ec79-4c23-80f2-92a840522039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:16.950039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08258111-ec79-4c23-80f2-92a840522039 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:16.950739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08258111-ec79-4c23-80f2-92a840522039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:16.951704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08258111-ec79-4c23-80f2-92a840522039 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:16.952572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08258111-ec79-4c23-80f2-92a840522039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:16.955668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08258111-ec79-4c23-80f2-92a840522039 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:16.975501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 648/1297] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:16 2022] PUT /identity/v3/projects/770206629c1d4eb89e5ddf603e7708eb/users/edcc51c2df9e4c0aa71581b0c2f74f80/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:16.976580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55071b67-6c20-464c-b9a4-f8f659e3c29c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:16.997680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 650/1298] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/users => generated 308 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:17.001030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08258111-ec79-4c23-80f2-92a840522039 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:17.027462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08258111-ec79-4c23-80f2-92a840522039 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:17.125092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55071b67-6c20-464c-b9a4-f8f659e3c29c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:17.138089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acab97-fe84-4465-9ca9-f9ee98c98584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:17.139167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acab97-fe84-4465-9ca9-f9ee98c98584 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.139457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acab97-fe84-4465-9ca9-f9ee98c98584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.139775 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acab97-fe84-4465-9ca9-f9ee98c98584 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.141142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acab97-fe84-4465-9ca9-f9ee98c98584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:17.146947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acab97-fe84-4465-9ca9-f9ee98c98584 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:17.192060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35bf5c-6697-440a-a3fd-e7431d244853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:17.193244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35bf5c-6697-440a-a3fd-e7431d244853 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.193967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35bf5c-6697-440a-a3fd-e7431d244853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.194614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35bf5c-6697-440a-a3fd-e7431d244853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.195509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35bf5c-6697-440a-a3fd-e7431d244853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:17.197740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35bf5c-6697-440a-a3fd-e7431d244853 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:17.201192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 649/1299] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:16 2022] POST /identity/v3/projects => generated 365 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:17.201192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf040d-5847-4cd9-b084-28694eb1b2d2 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.201192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf040d-5847-4cd9-b084-28694eb1b2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.201192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf040d-5847-4cd9-b084-28694eb1b2d2 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.208471 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 650/1300] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 24 13:12:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:12:17.209590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf040d-5847-4cd9-b084-28694eb1b2d2 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:17.218147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 651/1301] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:17.221769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35bf5c-6697-440a-a3fd-e7431d244853 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:17.252682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35bf5c-6697-440a-a3fd-e7431d244853 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:17.309788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:17.320142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf040d-5847-4cd9-b084-28694eb1b2d2 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:17.329824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8166d0-1bed-4725-8fec-ac1ae29d19bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:17.331354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8166d0-1bed-4725-8fec-ac1ae29d19bc admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.331762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8166d0-1bed-4725-8fec-ac1ae29d19bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.332511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8166d0-1bed-4725-8fec-ac1ae29d19bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.333250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8166d0-1bed-4725-8fec-ac1ae29d19bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:17.335387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8166d0-1bed-4725-8fec-ac1ae29d19bc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:17.354256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 651/1302] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:17.356916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a0525-86d1-4c35-9115-f6da9767f202 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.357303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a0525-86d1-4c35-9115-f6da9767f202 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.357679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a0525-86d1-4c35-9115-f6da9767f202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.373826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e9778-abea-4faa-b728-1196d09a9f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:17.376856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e9778-abea-4faa-b728-1196d09a9f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.377818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e9778-abea-4faa-b728-1196d09a9f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.378289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e9778-abea-4faa-b728-1196d09a9f87 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.380822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e9778-abea-4faa-b728-1196d09a9f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:17.383121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e9778-abea-4faa-b728-1196d09a9f87 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:17.447316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1f9a0525-86d1-4c35-9115-f6da9767f202 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:17.448519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 652/1303] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:17 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:17.448728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1f9a0525-86d1-4c35-9115-f6da9767f202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:17.449636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e9778-abea-4faa-b728-1196d09a9f87 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:17.466770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e9778-abea-4faa-b728-1196d09a9f87 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:17.479843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a60bf-10a9-490f-9463-6cefee927f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:17.480707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a60bf-10a9-490f-9463-6cefee927f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.480984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a60bf-10a9-490f-9463-6cefee927f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.483464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a60bf-10a9-490f-9463-6cefee927f8d admin admin] PATH_INFO: `/v3/projects/770206629c1d4eb89e5ddf603e7708eb/users/edcc51c2df9e4c0aa71581b0c2f74f80/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.489963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a60bf-10a9-490f-9463-6cefee927f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=770206629c1d4eb89e5ddf603e7708eb, user_id=edcc51c2df9e4c0aa71581b0c2f74f80, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:17.495925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a60bf-10a9-490f-9463-6cefee927f8d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:17.532876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 653/1304] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:17 2022] PUT /identity/v3/projects/770206629c1d4eb89e5ddf603e7708eb/users/edcc51c2df9e4c0aa71581b0c2f74f80/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:17.532876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a60bf-10a9-490f-9463-6cefee927f8d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:17.548816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:17.599984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 652/1305] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 24 13:12:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:17.601411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a0525-86d1-4c35-9115-f6da9767f202 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:17.630371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a60bf-10a9-490f-9463-6cefee927f8d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:17.633060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a0525-86d1-4c35-9115-f6da9767f202 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:17.638550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2080f89-863a-4c13-99e6-3d82e4feb7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:17.639507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2080f89-863a-4c13-99e6-3d82e4feb7db admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.639789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2080f89-863a-4c13-99e6-3d82e4feb7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.640462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2080f89-863a-4c13-99e6-3d82e4feb7db admin admin] PATH_INFO: `/v3/projects/9bd280395e7e4b4fa3846922e19d6684/users/61ec1caf875647e39982b63ae9ab3133/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.645565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9659-b632-416a-9472-f29d3e2bff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:17.646213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9659-b632-416a-9472-f29d3e2bff23 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.649130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9659-b632-416a-9472-f29d3e2bff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.649434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9659-b632-416a-9472-f29d3e2bff23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.650116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c9659-b632-416a-9472-f29d3e2bff23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:17.654500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c9659-b632-416a-9472-f29d3e2bff23 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:17.660665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2080f89-863a-4c13-99e6-3d82e4feb7db admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd280395e7e4b4fa3846922e19d6684, user_id=61ec1caf875647e39982b63ae9ab3133, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:17.664746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2080f89-863a-4c13-99e6-3d82e4feb7db admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:17.670479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 653/1306] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:17.670992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1ba0-8866-49d0-923b-c8f4e049369e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.671932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1ba0-8866-49d0-923b-c8f4e049369e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.672302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c1ba0-8866-49d0-923b-c8f4e049369e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.695517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 654/1307] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:17 2022] PUT /identity/v3/projects/9bd280395e7e4b4fa3846922e19d6684/users/61ec1caf875647e39982b63ae9ab3133/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:17.699450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2080f89-863a-4c13-99e6-3d82e4feb7db admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:17.735445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-cc2c1ba0-8866-49d0-923b-c8f4e049369e None None] MFA Rules not processed for user `edcc51c2df9e4c0aa71581b0c2f74f80`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:17.738730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-cc2c1ba0-8866-49d0-923b-c8f4e049369e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:17.856163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2080f89-863a-4c13-99e6-3d82e4feb7db admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:17.873266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9a1e9-20a2-4c51-8b22-74964a93ee3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:17.876107 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a1e9-20a2-4c51-8b22-74964a93ee3b service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:17.877010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a1e9-20a2-4c51-8b22-74964a93ee3b service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:17.878560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9a1e9-20a2-4c51-8b22-74964a93ee3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:17.898156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 654/1308] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:17.900488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c1ba0-8866-49d0-923b-c8f4e049369e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:18.013746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:18.072545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c1ba0-8866-49d0-923b-c8f4e049369e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:18.076891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9a1e9-20a2-4c51-8b22-74964a93ee3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:18.079007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9a1e9-20a2-4c51-8b22-74964a93ee3b service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:18.081684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dc0c9-730f-430c-9e64-d684971d3ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:18.082772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc0c9-730f-430c-9e64-d684971d3ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.083083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc0c9-730f-430c-9e64-d684971d3ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.085754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8dc0c9-730f-430c-9e64-d684971d3ab3 admin admin] PATH_INFO: `/v3/projects/115397ad2fbf44d7a8316ef82509d4dd/users/f6d1f5beb6404090a8aaca0ff54a968a/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.103539 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 655/1309] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:18.104716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9a1e9-20a2-4c51-8b22-74964a93ee3b service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:18.121164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8dc0c9-730f-430c-9e64-d684971d3ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115397ad2fbf44d7a8316ef82509d4dd, user_id=f6d1f5beb6404090a8aaca0ff54a968a, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:18.126159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8dc0c9-730f-430c-9e64-d684971d3ab3 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:18.138535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9a1e9-20a2-4c51-8b22-74964a93ee3b service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:18.148870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 655/1310] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:17 2022] PUT /identity/v3/projects/115397ad2fbf44d7a8316ef82509d4dd/users/f6d1f5beb6404090a8aaca0ff54a968a/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:18.150698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dc0c9-730f-430c-9e64-d684971d3ab3 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:18.195812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:18.286739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8dc0c9-730f-430c-9e64-d684971d3ab3 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:18.292686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edef621-5fc2-47bb-98dc-8e349f959c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:18.293931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef621-5fc2-47bb-98dc-8e349f959c33 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.294453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef621-5fc2-47bb-98dc-8e349f959c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.294928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edef621-5fc2-47bb-98dc-8e349f959c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.299485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9141b-b79b-49b1-a04f-2fc37516b678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:18.300228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9141b-b79b-49b1-a04f-2fc37516b678 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.300616 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9141b-b79b-49b1-a04f-2fc37516b678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.301010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9141b-b79b-49b1-a04f-2fc37516b678 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.301742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9141b-b79b-49b1-a04f-2fc37516b678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:18.304227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9141b-b79b-49b1-a04f-2fc37516b678 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:18.334468 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 656/1311] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:18.335784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9141b-b79b-49b1-a04f-2fc37516b678 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:18.361308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9141b-b79b-49b1-a04f-2fc37516b678 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:18.372908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33ed20-92d0-480c-9c7c-39cc835ffa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:18.373688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33ed20-92d0-480c-9c7c-39cc835ffa16 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.374083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33ed20-92d0-480c-9c7c-39cc835ffa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.374453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33ed20-92d0-480c-9c7c-39cc835ffa16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.375549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33ed20-92d0-480c-9c7c-39cc835ffa16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:18.377263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33ed20-92d0-480c-9c7c-39cc835ffa16 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:18.404302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 657/1312] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:18.404704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33ed20-92d0-480c-9c7c-39cc835ffa16 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:18.433801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33ed20-92d0-480c-9c7c-39cc835ffa16 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:18.451191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98661d5f-db0d-4813-8f3a-98ed93def2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:18.452090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98661d5f-db0d-4813-8f3a-98ed93def2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.453955 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98661d5f-db0d-4813-8f3a-98ed93def2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.454331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98661d5f-db0d-4813-8f3a-98ed93def2d4 admin admin] PATH_INFO: `/v3/projects/9bd280395e7e4b4fa3846922e19d6684/users/61ec1caf875647e39982b63ae9ab3133/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.459384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98661d5f-db0d-4813-8f3a-98ed93def2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd280395e7e4b4fa3846922e19d6684, user_id=61ec1caf875647e39982b63ae9ab3133, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:18.464686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98661d5f-db0d-4813-8f3a-98ed93def2d4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:18.494126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edef621-5fc2-47bb-98dc-8e349f959c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:18.500507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edef621-5fc2-47bb-98dc-8e349f959c33 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:18.500721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 658/1313] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:18 2022] PUT /identity/v3/projects/9bd280395e7e4b4fa3846922e19d6684/users/61ec1caf875647e39982b63ae9ab3133/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:18.506240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98661d5f-db0d-4813-8f3a-98ed93def2d4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:18.543417 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 656/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:18.548063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f46b5c-f6dc-47df-adac-f3f7a9b2f93b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.548351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f46b5c-f6dc-47df-adac-f3f7a9b2f93b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.625450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f46b5c-f6dc-47df-adac-f3f7a9b2f93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.676269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c7f46b5c-f6dc-47df-adac-f3f7a9b2f93b None None] MFA Rules not processed for user `61ec1caf875647e39982b63ae9ab3133`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:18.679425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c7f46b5c-f6dc-47df-adac-f3f7a9b2f93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:18.747085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98661d5f-db0d-4813-8f3a-98ed93def2d4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:18.761682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a5024-677f-4173-af15-dd479589835d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:18.763082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5024-677f-4173-af15-dd479589835d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.782165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5024-677f-4173-af15-dd479589835d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.785090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5024-677f-4173-af15-dd479589835d admin admin] PATH_INFO: `/v3/projects/115397ad2fbf44d7a8316ef82509d4dd/users/f6d1f5beb6404090a8aaca0ff54a968a/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.791567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a5024-677f-4173-af15-dd479589835d admin admin] RBAC: Authorizing `identity:create_grant(project_id=115397ad2fbf44d7a8316ef82509d4dd, user_id=f6d1f5beb6404090a8aaca0ff54a968a, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:18.797677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a5024-677f-4173-af15-dd479589835d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:18.830811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 659/1315] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:18 2022] PUT /identity/v3/projects/115397ad2fbf44d7a8316ef82509d4dd/users/f6d1f5beb6404090a8aaca0ff54a968a/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:18.832647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a5024-677f-4173-af15-dd479589835d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:18.859847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 657/1316] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:18 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:18.870255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efe0c-a719-4bd4-a5f0-c514439e6687 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.870552 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efe0c-a719-4bd4-a5f0-c514439e6687 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.870834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efe0c-a719-4bd4-a5f0-c514439e6687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.888226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:18.931734 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a5024-677f-4173-af15-dd479589835d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:18.937824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d84e5-6236-4aa1-8e98-aaa3dc6d62e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:18.938631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d84e5-6236-4aa1-8e98-aaa3dc6d62e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:18.939204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d84e5-6236-4aa1-8e98-aaa3dc6d62e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:18.939587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d84e5-6236-4aa1-8e98-aaa3dc6d62e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:18.941706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d42efe0c-a719-4bd4-a5f0-c514439e6687 None None] MFA Rules not processed for user `f6d1f5beb6404090a8aaca0ff54a968a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:18.943197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d42efe0c-a719-4bd4-a5f0-c514439e6687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:18.997852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:19.025161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d84e5-6236-4aa1-8e98-aaa3dc6d62e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:19.029269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d84e5-6236-4aa1-8e98-aaa3dc6d62e5 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:19.043471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 660/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:19.044534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d84e5-6236-4aa1-8e98-aaa3dc6d62e5 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:19.077693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d84e5-6236-4aa1-8e98-aaa3dc6d62e5 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:19.088968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a8f52-6ab8-4f7f-b899-90f45f0557ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:19.090426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a8f52-6ab8-4f7f-b899-90f45f0557ea service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:19.090815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a8f52-6ab8-4f7f-b899-90f45f0557ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:19.091267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a8f52-6ab8-4f7f-b899-90f45f0557ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:19.118830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 658/1318] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:19.241945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a8f52-6ab8-4f7f-b899-90f45f0557ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:19.243576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a8f52-6ab8-4f7f-b899-90f45f0557ea service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:19.271302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 661/1319] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:19 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:19.271859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2dce5-b192-4f9b-b000-1a8fd7564554 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:19.272181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2dce5-b192-4f9b-b000-1a8fd7564554 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:19.272504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2dce5-b192-4f9b-b000-1a8fd7564554 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:19.274891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 659/1320] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 24 13:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:12:19.300568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490114e-b896-4ee5-93ca-9bac014549e8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:19.301042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490114e-b896-4ee5-93ca-9bac014549e8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:19.301421 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490114e-b896-4ee5-93ca-9bac014549e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:19.368188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f490114e-b896-4ee5-93ca-9bac014549e8 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:19.369890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f490114e-b896-4ee5-93ca-9bac014549e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:19.412710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2dce5-b192-4f9b-b000-1a8fd7564554 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:19.432362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2dce5-b192-4f9b-b000-1a8fd7564554 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:19.440434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046a9cf-6b9f-46b5-bcdd-00c10de36f63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:19.441980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a9cf-6b9f-46b5-bcdd-00c10de36f63 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:19.442383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a9cf-6b9f-46b5-bcdd-00c10de36f63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:19.442764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046a9cf-6b9f-46b5-bcdd-00c10de36f63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:19.463701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046a9cf-6b9f-46b5-bcdd-00c10de36f63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:19.469199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a046a9cf-6b9f-46b5-bcdd-00c10de36f63 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:19.541912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 660/1321] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:19.623619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 662/1322] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 24 13:12:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:19.630873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab4cf5-0c07-43c1-806b-51830ab08e04 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:19.631333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab4cf5-0c07-43c1-806b-51830ab08e04 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:19.631717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab4cf5-0c07-43c1-806b-51830ab08e04 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:19.633322 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 661/1323] 10.222.0.42 () {54 vars in 912 bytes} [Mon Oct 24 13:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:12:19.643651 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c56da4-4dd2-4047-aa65-55258faaab55 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:19.644069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c56da4-4dd2-4047-aa65-55258faaab55 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:19.644492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c56da4-4dd2-4047-aa65-55258faaab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:19.712172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e5c56da4-4dd2-4047-aa65-55258faaab55 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:19.721186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e5c56da4-4dd2-4047-aa65-55258faaab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:19.753221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 663/1324] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:12:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:19.755646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab4cf5-0c07-43c1-806b-51830ab08e04 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:19.768892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab4cf5-0c07-43c1-806b-51830ab08e04 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:19.773948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9aaa91-0176-46f0-a987-a7815c956920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:19.774565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9aaa91-0176-46f0-a987-a7815c956920 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:19.774957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9aaa91-0176-46f0-a987-a7815c956920 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:19.775465 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9aaa91-0176-46f0-a987-a7815c956920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:19.802189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c56da4-4dd2-4047-aa65-55258faaab55 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:19.823623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c56da4-4dd2-4047-aa65-55258faaab55 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:19.829583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c12d2f-cdba-4ba6-8b73-5199a5bff3bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:19.831224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c12d2f-cdba-4ba6-8b73-5199a5bff3bd service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:19.831688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c12d2f-cdba-4ba6-8b73-5199a5bff3bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:19.832894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c12d2f-cdba-4ba6-8b73-5199a5bff3bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:19.849604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c12d2f-cdba-4ba6-8b73-5199a5bff3bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:19.854751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c12d2f-cdba-4ba6-8b73-5199a5bff3bd service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:19.866656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9aaa91-0176-46f0-a987-a7815c956920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:19.868712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9aaa91-0176-46f0-a987-a7815c956920 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:19.873161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 664/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:12:19.902112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 662/1326] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:20.054038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c12d2f-cdba-4ba6-8b73-5199a5bff3bd service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:20.113520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272a786-b75c-4b91-984e-1bd0e1143169 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:20.114028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272a786-b75c-4b91-984e-1bd0e1143169 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:20.114493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272a786-b75c-4b91-984e-1bd0e1143169 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:20.116667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 663/1327] 10.222.0.42 () {54 vars in 912 bytes} [Mon Oct 24 13:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 24 13:12:20.130520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb6ea-5b03-4ec1-8bd9-5097be697581 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:20.131030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb6ea-5b03-4ec1-8bd9-5097be697581 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:20.131508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb6ea-5b03-4ec1-8bd9-5097be697581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:20.171112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f2cdb6ea-5b03-4ec1-8bd9-5097be697581 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:20.174137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f2cdb6ea-5b03-4ec1-8bd9-5097be697581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:20.182482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:20.206467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 664/1328] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:12:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:12:20.223445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdb6ea-5b03-4ec1-8bd9-5097be697581 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:20.226175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c12d2f-cdba-4ba6-8b73-5199a5bff3bd service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:20.242886 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:20.243581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:20.243794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdb6ea-5b03-4ec1-8bd9-5097be697581 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:20.243904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:20.244130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] PATH_INFO: `/v3/users/b3fd1f7c6b83402b8a7b7c7e5369acee` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:20.245289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3fd1f7c6b83402b8a7b7c7e5369acee)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:20.246435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:12:20.246435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Oct 24 13:12:20.246435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: Oct 24 13:12:20.247736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:20.254956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21745685-c590-4164-9ba4-c982fb2c253d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:20.256366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745685-c590-4164-9ba4-c982fb2c253d service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:20.256749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745685-c590-4164-9ba4-c982fb2c253d service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:20.257670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21745685-c590-4164-9ba4-c982fb2c253d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:20.294690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21745685-c590-4164-9ba4-c982fb2c253d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:20.302908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21745685-c590-4164-9ba4-c982fb2c253d service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:20.351085 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 665/1329] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:12:20.435351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3fd1f7c6b83402b8a7b7c7e5369acee'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.487808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3fd1f7c6b83402b8a7b7c7e5369acee'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.491449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3fd1f7c6b83402b8a7b7c7e5369acee'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.515799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3fd1f7c6b83402b8a7b7c7e5369acee'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.516430 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3fd1f7c6b83402b8a7b7c7e5369acee'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.547098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 665/1330] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:20 2022] DELETE /identity/v3/users/b3fd1f7c6b83402b8a7b7c7e5369acee => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:20.552577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f21bc-2b66-402b-b7c6-811ebfdb1f90 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:20.552858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f21bc-2b66-402b-b7c6-811ebfdb1f90 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:20.553140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f21bc-2b66-402b-b7c6-811ebfdb1f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:20.587221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:20.589954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:20.611122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-740f21bc-2b66-402b-b7c6-811ebfdb1f90 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:20.612937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-740f21bc-2b66-402b-b7c6-811ebfdb1f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:20.697033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:20.722768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 666/1331] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:20.735545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798266e0-55d5-4368-8699-f0633a155ae9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:20.741930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:20.741930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:20.741930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:20.744693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] PATH_INFO: `/v3/users/b8c8e7a6f6a9447a8c24dad629dff6a6` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:20.755236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c8e7a6f6a9447a8c24dad629dff6a6)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:20.759720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:20.795782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f21bc-2b66-402b-b7c6-811ebfdb1f90 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:20.799471 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-740f21bc-2b66-402b-b7c6-811ebfdb1f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:20.875218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c8e7a6f6a9447a8c24dad629dff6a6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.902395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c8e7a6f6a9447a8c24dad629dff6a6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.904175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c8e7a6f6a9447a8c24dad629dff6a6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.924620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c8e7a6f6a9447a8c24dad629dff6a6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.925341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73d915cf-7acc-4a5a-b28f-764774214c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c8e7a6f6a9447a8c24dad629dff6a6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:20.927127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:20.953462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 666/1332] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:20 2022] DELETE /identity/v3/users/b8c8e7a6f6a9447a8c24dad629dff6a6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:20.956181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee696-4fee-4d6e-b1cb-eedb0379a40b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:20.956561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee696-4fee-4d6e-b1cb-eedb0379a40b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:20.956845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caee696-4fee-4d6e-b1cb-eedb0379a40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:20.960187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f21bc-2b66-402b-b7c6-811ebfdb1f90 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:20.995614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4caee696-4fee-4d6e-b1cb-eedb0379a40b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:20.997427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4caee696-4fee-4d6e-b1cb-eedb0379a40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:21.113080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:21.114273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:21.115536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:21.115923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:21.119382 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:21.121103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 667/1333] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:21.145033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caee696-4fee-4d6e-b1cb-eedb0379a40b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:21.147967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4caee696-4fee-4d6e-b1cb-eedb0379a40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:21.154706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:21.156685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:21.182676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caee696-4fee-4d6e-b1cb-eedb0379a40b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:21.184587 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 667/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:21.195365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a8015a-5601-49dc-8756-a14f9fb1900e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:21.196005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8015a-5601-49dc-8756-a14f9fb1900e service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:21.196292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8015a-5601-49dc-8756-a14f9fb1900e service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:21.196571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a8015a-5601-49dc-8756-a14f9fb1900e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:21.215799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8015a-5601-49dc-8756-a14f9fb1900e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:21.219962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a8015a-5601-49dc-8756-a14f9fb1900e service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:21.244240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 668/1335] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:24.067419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:24.069852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:24.099618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01738d-02b5-40b0-8b68-7e443e10001a service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:24.132436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:24.135605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:24.138218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:24.138839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] PATH_INFO: `/v3/projects/54de867d20774b7c9d90e8f06795a918` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:24.143658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54de867d20774b7c9d90e8f06795a918)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:24.145261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: WARNING py.warnings [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 24 13:12:24.145261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: warnings.warn(msg) Oct 24 13:12:24.145261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: Oct 24 13:12:24.146919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:24.333086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54de867d20774b7c9d90e8f06795a918', 'request_id': 'req-3e3e8e7e-bca0-4191-8928-ff9b002445f1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.383577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54de867d20774b7c9d90e8f06795a918', 'request_id': 'req-3e3e8e7e-bca0-4191-8928-ff9b002445f1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.408824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54de867d20774b7c9d90e8f06795a918', 'request_id': 'req-3e3e8e7e-bca0-4191-8928-ff9b002445f1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.451869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] The token cache is being invalidate because project 54de867d20774b7c9d90e8f06795a918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:24.452171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.457310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.464336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 668/1336] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:24 2022] DELETE /identity/v3/projects/54de867d20774b7c9d90e8f06795a918 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:24.470589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089e9e-857c-47bc-a02b-1c50f3a1abf5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:24.471188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089e9e-857c-47bc-a02b-1c50f3a1abf5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:24.471540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19089e9e-857c-47bc-a02b-1c50f3a1abf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:24.486985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:24.489298 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:24.535179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-19089e9e-857c-47bc-a02b-1c50f3a1abf5 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:24.537436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-19089e9e-857c-47bc-a02b-1c50f3a1abf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:24.675184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:24.747542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 669/1337] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:24.750150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bdb96-b16a-40ac-823f-6fd4008a14f2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:24.751601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e8e7e-bca0-4191-8928-ff9b002445f1 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:24.753262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bdb96-b16a-40ac-823f-6fd4008a14f2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:24.754120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bdb96-b16a-40ac-823f-6fd4008a14f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:24.764914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:24.767332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:24.767481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:24.771702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] PATH_INFO: `/v3/projects/07c5c1b5f3cb43e39ac7db457f1189aa` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:24.771702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c5c1b5f3cb43e39ac7db457f1189aa)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:24.776070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:24.802023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-509bdb96-b16a-40ac-823f-6fd4008a14f2 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:24.806335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-509bdb96-b16a-40ac-823f-6fd4008a14f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:24.830425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 670/1338] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:24.835097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bdb96-b16a-40ac-823f-6fd4008a14f2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:24.863085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509bdb96-b16a-40ac-823f-6fd4008a14f2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:24.871977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:24.872947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:24.876612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:24.877646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:24.880248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:24.880750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:24.886363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:24.892240 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c5c1b5f3cb43e39ac7db457f1189aa', 'request_id': 'req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.909817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c5c1b5f3cb43e39ac7db457f1189aa', 'request_id': 'req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.929107 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 671/1339] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:24.933688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:24.940061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c5c1b5f3cb43e39ac7db457f1189aa', 'request_id': 'req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.970061 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a3108-d622-4f31-9156-fb4e7dc6ae18 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:24.978827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] The token cache is being invalidate because project 07c5c1b5f3cb43e39ac7db457f1189aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:24.979145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.979824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:24.985395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 669/1340] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:24 2022] DELETE /identity/v3/projects/07c5c1b5f3cb43e39ac7db457f1189aa => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:24.986298 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:24.987710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:25.096548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:25.096729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d16beb-d0ab-48f1-ba2b-e511da92dc72 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:25.107925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:25.110418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:25.111563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:25.111858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:25.112214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] PATH_INFO: `/v3/users/61ec1caf875647e39982b63ae9ab3133` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:25.113086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:25.114752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:25.115714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:25.115907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ec1caf875647e39982b63ae9ab3133)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:25.116725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:25.120041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:25.122348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:25.180808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 672/1341] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:24 2022] POST /identity/v3/projects => generated 403 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:25.185911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:25.189139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:25.229410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ec1caf875647e39982b63ae9ab3133'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:25.250112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ec1caf875647e39982b63ae9ab3133'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:25.251304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ec1caf875647e39982b63ae9ab3133'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:25.255806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:25.270936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ec1caf875647e39982b63ae9ab3133'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:25.288019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ec1caf875647e39982b63ae9ab3133'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:25.296104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481d7c4-0e74-4303-922d-e72e6a7f7bbb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:25.303583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5a12d-dd42-4eab-941b-57a58b3e7e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:25.304596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5a12d-dd42-4eab-941b-57a58b3e7e31 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:25.305590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5a12d-dd42-4eab-941b-57a58b3e7e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:25.306209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a5a12d-dd42-4eab-941b-57a58b3e7e31 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:25.307026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5a12d-dd42-4eab-941b-57a58b3e7e31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:25.310259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 670/1342] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:24 2022] DELETE /identity/v3/users/61ec1caf875647e39982b63ae9ab3133 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:25.314479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a5a12d-dd42-4eab-941b-57a58b3e7e31 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:25.351362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 673/1343] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:25 2022] POST /identity/v3/users => generated 302 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:25.352101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:25.354535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:25.518178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7ac76-789f-4c99-87e5-fc37bea1de76 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:25.548789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba46cb4-ebf0-43d6-ba24-7d0e57d6d6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:25.549753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba46cb4-ebf0-43d6-ba24-7d0e57d6d6cc admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:25.550192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba46cb4-ebf0-43d6-ba24-7d0e57d6d6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:25.552007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba46cb4-ebf0-43d6-ba24-7d0e57d6d6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:25.553260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba46cb4-ebf0-43d6-ba24-7d0e57d6d6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:25.555716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba46cb4-ebf0-43d6-ba24-7d0e57d6d6cc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:25.583207 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 671/1344] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:12:25.585751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5a12d-dd42-4eab-941b-57a58b3e7e31 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:25.604108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a5a12d-dd42-4eab-941b-57a58b3e7e31 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:25.616486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25bd3d-9d2e-407c-9dc8-87a18eca5b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:25.619398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25bd3d-9d2e-407c-9dc8-87a18eca5b56 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:25.620990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25bd3d-9d2e-407c-9dc8-87a18eca5b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:25.621344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25bd3d-9d2e-407c-9dc8-87a18eca5b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:25.625132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25bd3d-9d2e-407c-9dc8-87a18eca5b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:25.627084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e25bd3d-9d2e-407c-9dc8-87a18eca5b56 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:25.963539 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba46cb4-ebf0-43d6-ba24-7d0e57d6d6cc admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:25.964575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 674/1345] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:26.014804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba46cb4-ebf0-43d6-ba24-7d0e57d6d6cc admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:26.032133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:26.035342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:26.036736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:26.037183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] PATH_INFO: `/v3/projects/c55ee1329da34912b0cb4df30bc11d2c/users/f443a8f611904179b1b3c425545265f4/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:26.093608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c55ee1329da34912b0cb4df30bc11d2c, user_id=f443a8f611904179b1b3c425545265f4, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:26.100386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:26.138415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 672/1346] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:25 2022] PUT /identity/v3/projects/c55ee1329da34912b0cb4df30bc11d2c/users/f443a8f611904179b1b3c425545265f4/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:26.144349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96363258-ee3e-4ea6-aaba-ac07c986441f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:26.144654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96363258-ee3e-4ea6-aaba-ac07c986441f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:26.144936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96363258-ee3e-4ea6-aaba-ac07c986441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:26.184340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-96363258-ee3e-4ea6-aaba-ac07c986441f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:26.185872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-96363258-ee3e-4ea6-aaba-ac07c986441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:26.404253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 675/1347] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:26.475119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:26.479750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:26.582454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474dc5e-9696-43f4-90d8-279f62db4ed5 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:26.591512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce97408-c530-4642-8a5b-57594590b9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:26.592164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce97408-c530-4642-8a5b-57594590b9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:26.592500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce97408-c530-4642-8a5b-57594590b9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:26.592779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce97408-c530-4642-8a5b-57594590b9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:26.605731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce97408-c530-4642-8a5b-57594590b9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:26.607405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce97408-c530-4642-8a5b-57594590b9e5 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:26.628873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 673/1348] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:26.912446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96363258-ee3e-4ea6-aaba-ac07c986441f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:26.914035 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-96363258-ee3e-4ea6-aaba-ac07c986441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:26.935581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96363258-ee3e-4ea6-aaba-ac07c986441f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:26.947920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:26.948585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:26.948888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:26.949198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] PATH_INFO: `/v3/projects/9bd280395e7e4b4fa3846922e19d6684` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:26.953769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd280395e7e4b4fa3846922e19d6684)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:26.955971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:27.112074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd280395e7e4b4fa3846922e19d6684', 'request_id': 'req-cf919f29-458f-419f-b277-3600be06172e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:27.122112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd280395e7e4b4fa3846922e19d6684', 'request_id': 'req-cf919f29-458f-419f-b277-3600be06172e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:27.132537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:27.148823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd280395e7e4b4fa3846922e19d6684', 'request_id': 'req-cf919f29-458f-419f-b277-3600be06172e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:27.172480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] The token cache is being invalidate because project 9bd280395e7e4b4fa3846922e19d6684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:27.172817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:27.173882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cf919f29-458f-419f-b277-3600be06172e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:27.176717 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 676/1349] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:26 2022] DELETE /identity/v3/projects/9bd280395e7e4b4fa3846922e19d6684 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:27.181375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b1e28-01f2-43b6-8610-1c173afce30b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:27.181838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b1e28-01f2-43b6-8610-1c173afce30b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:27.181919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b1e28-01f2-43b6-8610-1c173afce30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:27.250438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-de6b1e28-01f2-43b6-8610-1c173afce30b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:27.251659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-de6b1e28-01f2-43b6-8610-1c173afce30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:27.447313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:27.477895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 674/1350] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:27.488460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf041a-e0c6-4684-8b47-1316df7d9895 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:27.488772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf041a-e0c6-4684-8b47-1316df7d9895 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:27.489097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf041a-e0c6-4684-8b47-1316df7d9895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:27.526464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-abdf041a-e0c6-4684-8b47-1316df7d9895 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:27.531466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-abdf041a-e0c6-4684-8b47-1316df7d9895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:27.558188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 677/1351] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:27.563545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b1e28-01f2-43b6-8610-1c173afce30b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:27.579740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b1e28-01f2-43b6-8610-1c173afce30b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:27.591313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:27.592140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:27.592621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:27.593023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:27.597968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:27.598904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:27.603650 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:27.632374 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 675/1352] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:27.637383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf041a-e0c6-4684-8b47-1316df7d9895 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:27.653725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf041a-e0c6-4684-8b47-1316df7d9895 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:27.667776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa1776-932c-47cb-9591-315db02a3700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:27.668414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa1776-932c-47cb-9591-315db02a3700 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:27.668700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa1776-932c-47cb-9591-315db02a3700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:27.668981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa1776-932c-47cb-9591-315db02a3700 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:27.669692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa1776-932c-47cb-9591-315db02a3700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:27.672179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa1776-932c-47cb-9591-315db02a3700 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:27.727752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 678/1353] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:27 2022] POST /identity/v3/projects => generated 387 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:27.731514 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:27.747514 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb71069-1c29-479d-a3bd-64179979822b admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:27.945074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdfff-dbfa-4ba2-bb42-da13510f375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:27.945992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdfff-dbfa-4ba2-bb42-da13510f375c admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:27.946302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdfff-dbfa-4ba2-bb42-da13510f375c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:27.946699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cdfff-dbfa-4ba2-bb42-da13510f375c admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:27.947434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cdfff-dbfa-4ba2-bb42-da13510f375c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:27.949426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cdfff-dbfa-4ba2-bb42-da13510f375c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:28.039993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 676/1354] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:27 2022] POST /identity/v3/users => generated 294 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:28.044111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa1776-932c-47cb-9591-315db02a3700 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:28.069166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa1776-932c-47cb-9591-315db02a3700 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:28.088008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddc394-9cd1-4a2b-a77d-4be7ca1f9a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:28.088979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc394-9cd1-4a2b-a77d-4be7ca1f9a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:28.089799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc394-9cd1-4a2b-a77d-4be7ca1f9a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:28.090522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ddc394-9cd1-4a2b-a77d-4be7ca1f9a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:28.092145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddc394-9cd1-4a2b-a77d-4be7ca1f9a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:28.097223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ddc394-9cd1-4a2b-a77d-4be7ca1f9a23 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:28.143316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 679/1355] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:28.143982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdfff-dbfa-4ba2-bb42-da13510f375c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:28.162550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cdfff-dbfa-4ba2-bb42-da13510f375c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:28.183822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29301f-0f2c-4c43-9b83-552164565baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:28.185364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29301f-0f2c-4c43-9b83-552164565baa admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:28.186846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29301f-0f2c-4c43-9b83-552164565baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:28.187315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29301f-0f2c-4c43-9b83-552164565baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:28.188246 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29301f-0f2c-4c43-9b83-552164565baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:28.196503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29301f-0f2c-4c43-9b83-552164565baa admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:28.234226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 677/1356] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:28.235224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddc394-9cd1-4a2b-a77d-4be7ca1f9a23 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:28.245341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ddc394-9cd1-4a2b-a77d-4be7ca1f9a23 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:28.273408 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b113996-6dce-49f1-8e89-d30627df4ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:28.274101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b113996-6dce-49f1-8e89-d30627df4ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:28.274651 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b113996-6dce-49f1-8e89-d30627df4ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:28.274651 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b113996-6dce-49f1-8e89-d30627df4ce4 admin admin] PATH_INFO: `/v3/projects/6286671a05ed4748baec43c2c31bd089/users/67faa0aaf70d4f9c90c5581bceab1c08/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:28.320106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b113996-6dce-49f1-8e89-d30627df4ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6286671a05ed4748baec43c2c31bd089, user_id=67faa0aaf70d4f9c90c5581bceab1c08, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:28.322799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b113996-6dce-49f1-8e89-d30627df4ce4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:28.377175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 680/1357] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:28 2022] PUT /identity/v3/projects/6286671a05ed4748baec43c2c31bd089/users/67faa0aaf70d4f9c90c5581bceab1c08/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:28.379506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30fcba-b671-4624-b41a-32f1f33a898a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:28.379506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30fcba-b671-4624-b41a-32f1f33a898a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:28.379506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c30fcba-b671-4624-b41a-32f1f33a898a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:28.441067 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7c30fcba-b671-4624-b41a-32f1f33a898a None None] MFA Rules not processed for user `67faa0aaf70d4f9c90c5581bceab1c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:28.444061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c30fcba-b671-4624-b41a-32f1f33a898a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:28.640939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 678/1358] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:12:28.644967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b113996-6dce-49f1-8e89-d30627df4ce4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:28.772423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b113996-6dce-49f1-8e89-d30627df4ce4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:28.782757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fa9e6-26d0-4879-8b62-acd1c17269fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:28.785645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fa9e6-26d0-4879-8b62-acd1c17269fc admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:28.786033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fa9e6-26d0-4879-8b62-acd1c17269fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:28.786578 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33fa9e6-26d0-4879-8b62-acd1c17269fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:28.788560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fa9e6-26d0-4879-8b62-acd1c17269fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:28.796976 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33fa9e6-26d0-4879-8b62-acd1c17269fc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:28.865213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 681/1359] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:28 2022] POST /identity/v3/projects => generated 387 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:28.869419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30fcba-b671-4624-b41a-32f1f33a898a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:28.885713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c30fcba-b671-4624-b41a-32f1f33a898a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:29.119589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:29.145198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10def953-04e6-440e-8786-d0c5585de8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:29.145979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10def953-04e6-440e-8786-d0c5585de8dd admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:29.146333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10def953-04e6-440e-8786-d0c5585de8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:29.146683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10def953-04e6-440e-8786-d0c5585de8dd admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:29.147453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10def953-04e6-440e-8786-d0c5585de8dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:29.149598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10def953-04e6-440e-8786-d0c5585de8dd admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:29.223795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 679/1360] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:28 2022] POST /identity/v3/users => generated 300 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:29.228832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fa9e6-26d0-4879-8b62-acd1c17269fc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:29.257993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33fa9e6-26d0-4879-8b62-acd1c17269fc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:29.273334 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a11d44-1103-4931-9758-57223d2de640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:29.275692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a11d44-1103-4931-9758-57223d2de640 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:29.279293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a11d44-1103-4931-9758-57223d2de640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:29.279614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a11d44-1103-4931-9758-57223d2de640 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:29.282105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a11d44-1103-4931-9758-57223d2de640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:29.294553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a11d44-1103-4931-9758-57223d2de640 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:29.315206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 682/1361] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:29.319385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10def953-04e6-440e-8786-d0c5585de8dd admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:29.338967 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10def953-04e6-440e-8786-d0c5585de8dd admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:29.358217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b239c0-a49e-4dca-94da-92be11b9481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:29.360410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b239c0-a49e-4dca-94da-92be11b9481d admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:29.360698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b239c0-a49e-4dca-94da-92be11b9481d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:29.360980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b239c0-a49e-4dca-94da-92be11b9481d admin admin] PATH_INFO: `/v3/projects/3b08d3a7d9be417dab442e1b9363c751/users/71896fc97c304b2e8454b2053c2f86ea/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:29.423555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b239c0-a49e-4dca-94da-92be11b9481d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b08d3a7d9be417dab442e1b9363c751, user_id=71896fc97c304b2e8454b2053c2f86ea, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:29.426273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b239c0-a49e-4dca-94da-92be11b9481d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:29.463312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 680/1362] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:29 2022] PUT /identity/v3/projects/3b08d3a7d9be417dab442e1b9363c751/users/71896fc97c304b2e8454b2053c2f86ea/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:29.467837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a11d44-1103-4931-9758-57223d2de640 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:29.484697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a11d44-1103-4931-9758-57223d2de640 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:29.666028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:29.689689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a96c2-112b-4823-9cc7-eb4563ba94ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:29.690436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a96c2-112b-4823-9cc7-eb4563ba94ff admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:29.690826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a96c2-112b-4823-9cc7-eb4563ba94ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:29.691234 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a96c2-112b-4823-9cc7-eb4563ba94ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:29.691997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a96c2-112b-4823-9cc7-eb4563ba94ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:29.693849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a96c2-112b-4823-9cc7-eb4563ba94ff admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:29.756494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 683/1363] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:29.760563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b239c0-a49e-4dca-94da-92be11b9481d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:29.782874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b239c0-a49e-4dca-94da-92be11b9481d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:29.799741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:29.800855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:29.801143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:29.801423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] PATH_INFO: `/v3/projects/3b08d3a7d9be417dab442e1b9363c751/users/71896fc97c304b2e8454b2053c2f86ea/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:29.804932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b08d3a7d9be417dab442e1b9363c751, user_id=71896fc97c304b2e8454b2053c2f86ea, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:29.811044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:29.863216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 681/1364] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:29 2022] PUT /identity/v3/projects/3b08d3a7d9be417dab442e1b9363c751/users/71896fc97c304b2e8454b2053c2f86ea/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:29.874949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3f72c-2c26-4279-9bfc-6fa92a365339 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:29.875828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3f72c-2c26-4279-9bfc-6fa92a365339 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:29.875828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3f72c-2c26-4279-9bfc-6fa92a365339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:29.918467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-23c3f72c-2c26-4279-9bfc-6fa92a365339 None None] MFA Rules not processed for user `71896fc97c304b2e8454b2053c2f86ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:29.919720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-23c3f72c-2c26-4279-9bfc-6fa92a365339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:30.043652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 684/1365] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:30.056723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:30.058835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:30.198414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c793dce-1f3f-42ef-ba8a-81e1122e7044 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:30.203159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbff870-34c5-4e09-b5e3-d967ba69269f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:30.204156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbff870-34c5-4e09-b5e3-d967ba69269f service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:30.204615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbff870-34c5-4e09-b5e3-d967ba69269f service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:30.205050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbff870-34c5-4e09-b5e3-d967ba69269f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:30.229792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbff870-34c5-4e09-b5e3-d967ba69269f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:30.232086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbff870-34c5-4e09-b5e3-d967ba69269f service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:30.269131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 682/1366] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:30.318504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3f72c-2c26-4279-9bfc-6fa92a365339 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:30.610479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3f72c-2c26-4279-9bfc-6fa92a365339 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:30.619861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:30.620669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:30.621172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:30.621603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] PATH_INFO: `/v3/users/67faa0aaf70d4f9c90c5581bceab1c08` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:30.625526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67faa0aaf70d4f9c90c5581bceab1c08)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:30.631555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:30.651994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7f51-f1a9-45b5-9d5f-5d4ecf94ce49 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:30.652284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7f51-f1a9-45b5-9d5f-5d4ecf94ce49 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:30.652569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7f51-f1a9-45b5-9d5f-5d4ecf94ce49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:30.721502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d85c7f51-f1a9-45b5-9d5f-5d4ecf94ce49 None None] MFA Rules not processed for user `b3e637d319f449f38e4d6375ae5b22fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:30.723272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d85c7f51-f1a9-45b5-9d5f-5d4ecf94ce49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:30.763269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67faa0aaf70d4f9c90c5581bceab1c08'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:30.790851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67faa0aaf70d4f9c90c5581bceab1c08'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:30.792179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67faa0aaf70d4f9c90c5581bceab1c08'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:30.818371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67faa0aaf70d4f9c90c5581bceab1c08'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:30.819220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19ab9586-f77a-4fa3-b0f5-ba25e729ed70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67faa0aaf70d4f9c90c5581bceab1c08'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:30.835989 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:30.855596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 685/1367] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:30 2022] DELETE /identity/v3/users/67faa0aaf70d4f9c90c5581bceab1c08 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:30.862461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818cc36-59c3-443c-8386-53d42c316d27 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:30.863331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818cc36-59c3-443c-8386-53d42c316d27 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:30.863693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818cc36-59c3-443c-8386-53d42c316d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:30.887631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 683/1368] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:30.912133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4fcb-1c56-434e-bf9b-d0581bb19cdc None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:30.919179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4fcb-1c56-434e-bf9b-d0581bb19cdc None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:30.919546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4fcb-1c56-434e-bf9b-d0581bb19cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:30.979624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a86d4fcb-1c56-434e-bf9b-d0581bb19cdc None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:30.981247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a86d4fcb-1c56-434e-bf9b-d0581bb19cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:30.993291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b818cc36-59c3-443c-8386-53d42c316d27 None None] MFA Rules not processed for user `d29ff25b062346f6bcf9910b1fa8db5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:30.998854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b818cc36-59c3-443c-8386-53d42c316d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:31.122024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:31.180883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 684/1369] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:31.182441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a753ef-f0e4-4c85-adfd-de8fcb9b6c4e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:31.182832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a753ef-f0e4-4c85-adfd-de8fcb9b6c4e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:31.183459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a753ef-f0e4-4c85-adfd-de8fcb9b6c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:31.377506 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 686/1370] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 511 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:31.378070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-92a753ef-f0e4-4c85-adfd-de8fcb9b6c4e None None] MFA Rules not processed for user `f443a8f611904179b1b3c425545265f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:31.378446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d9fa0-ef8d-4a5b-b408-ae3053459dda None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:31.378446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d9fa0-ef8d-4a5b-b408-ae3053459dda None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:31.378446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d9fa0-ef8d-4a5b-b408-ae3053459dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:31.378815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-92a753ef-f0e4-4c85-adfd-de8fcb9b6c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:31.485856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-814d9fa0-ef8d-4a5b-b408-ae3053459dda None None] MFA Rules not processed for user `b3e637d319f449f38e4d6375ae5b22fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:31.487255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-814d9fa0-ef8d-4a5b-b408-ae3053459dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:31.699958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 685/1371] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 518 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:31.702590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a753ef-f0e4-4c85-adfd-de8fcb9b6c4e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:31.705810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-92a753ef-f0e4-4c85-adfd-de8fcb9b6c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:31.716181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 687/1372] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:31.717608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a246bc-5403-4ec6-bffe-bc18f0ceb999 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:31.717959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a246bc-5403-4ec6-bffe-bc18f0ceb999 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:31.735860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a246bc-5403-4ec6-bffe-bc18f0ceb999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:31.836829 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-21a246bc-5403-4ec6-bffe-bc18f0ceb999 None None] MFA Rules not processed for user `d29ff25b062346f6bcf9910b1fa8db5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:31.838080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-21a246bc-5403-4ec6-bffe-bc18f0ceb999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:31.882565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 688/1373] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:31.886920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a246bc-5403-4ec6-bffe-bc18f0ceb999 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:31.909200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-21a246bc-5403-4ec6-bffe-bc18f0ceb999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:31.960266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a246bc-5403-4ec6-bffe-bc18f0ceb999 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:31.975270 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a753ef-f0e4-4c85-adfd-de8fcb9b6c4e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:31.980782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:31.981934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:31.984293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:31.985052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:31.988901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:31.988901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:31.988901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:31.989438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:31.991389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:31.994348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:32.022685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:32.025106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:32.069729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 686/1374] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:32.071540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 689/1375] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:31 2022] POST /identity/v3/projects => generated 405 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:32.074444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:32.076634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:32.078640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:32.084420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:32.339415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:32.383039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605eb00f-9026-4529-951f-3d320f63e6f3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:32.392287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e417280-32f4-4bc4-92bd-d75b440ebf35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:32.392945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c6f3b-1841-4417-9853-e6d9a5157441 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:32.393396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e417280-32f4-4bc4-92bd-d75b440ebf35 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:32.393396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e417280-32f4-4bc4-92bd-d75b440ebf35 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:32.393841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e417280-32f4-4bc4-92bd-d75b440ebf35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:32.417148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3b17e-7fde-4fe9-9604-517244490193 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:32.418721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3b17e-7fde-4fe9-9604-517244490193 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:32.419025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3b17e-7fde-4fe9-9604-517244490193 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:32.424821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3b17e-7fde-4fe9-9604-517244490193 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:32.456245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:32.533317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e417280-32f4-4bc4-92bd-d75b440ebf35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:32.535515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e417280-32f4-4bc4-92bd-d75b440ebf35 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:32.553098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 690/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:32.554437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e417280-32f4-4bc4-92bd-d75b440ebf35 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:32.576295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3b17e-7fde-4fe9-9604-517244490193 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:32.579431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3b17e-7fde-4fe9-9604-517244490193 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:32.606623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 687/1377] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:32.608446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960256a-588c-4241-8cd5-adbf3ef1e1dd None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:32.609856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960256a-588c-4241-8cd5-adbf3ef1e1dd None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:32.610345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960256a-588c-4241-8cd5-adbf3ef1e1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:32.716981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e417280-32f4-4bc4-92bd-d75b440ebf35 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:32.722202 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4960256a-588c-4241-8cd5-adbf3ef1e1dd None None] MFA Rules not processed for user `c5619c8183e04e1d9073f1b40e057967`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:32.724211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4960256a-588c-4241-8cd5-adbf3ef1e1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:32.742364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529649d1-e333-4492-9abf-8bbbbf6933ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:32.743898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529649d1-e333-4492-9abf-8bbbbf6933ea admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:32.747698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529649d1-e333-4492-9abf-8bbbbf6933ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:32.748083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529649d1-e333-4492-9abf-8bbbbf6933ea admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:32.754347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529649d1-e333-4492-9abf-8bbbbf6933ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:32.760741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529649d1-e333-4492-9abf-8bbbbf6933ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:32.841556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 691/1378] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:32 2022] POST /identity/v3/users => generated 309 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:32.848735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529649d1-e333-4492-9abf-8bbbbf6933ea admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:32.867535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529649d1-e333-4492-9abf-8bbbbf6933ea admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:32.893235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624e64c-7704-4bbd-b853-85cb060d973e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:32.894103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624e64c-7704-4bbd-b853-85cb060d973e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:32.895271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624e64c-7704-4bbd-b853-85cb060d973e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:32.895739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624e64c-7704-4bbd-b853-85cb060d973e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:32.896593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624e64c-7704-4bbd-b853-85cb060d973e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:32.898624 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624e64c-7704-4bbd-b853-85cb060d973e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:32.926852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 692/1379] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:32.928144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624e64c-7704-4bbd-b853-85cb060d973e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:32.971456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 688/1380] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:32.974013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624e64c-7704-4bbd-b853-85cb060d973e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:32.991339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e633f6-d2cc-4a0d-a5c5-bf57cfe0a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:32.991611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e633f6-d2cc-4a0d-a5c5-bf57cfe0a188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:32.991878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e633f6-d2cc-4a0d-a5c5-bf57cfe0a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:32.992146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e633f6-d2cc-4a0d-a5c5-bf57cfe0a188 admin admin] PATH_INFO: `/v3/projects/a96b5da347f744c9b305f206acb396fe/users/aaf859d57bfe47bdb28d8c44b2d52137/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:33.035970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e633f6-d2cc-4a0d-a5c5-bf57cfe0a188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96b5da347f744c9b305f206acb396fe, user_id=aaf859d57bfe47bdb28d8c44b2d52137, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:33.050765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e633f6-d2cc-4a0d-a5c5-bf57cfe0a188 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:33.096585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 693/1381] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:32 2022] PUT /identity/v3/projects/a96b5da347f744c9b305f206acb396fe/users/aaf859d57bfe47bdb28d8c44b2d52137/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:33.099096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960256a-588c-4241-8cd5-adbf3ef1e1dd None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:33.114680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960256a-588c-4241-8cd5-adbf3ef1e1dd None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:33.287879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:33.289857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:33.290151 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:33.290505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:33.291143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:33.300033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:33.329902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e633f6-d2cc-4a0d-a5c5-bf57cfe0a188 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:33.337757 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 689/1382] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:33 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:33.349478 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e633f6-d2cc-4a0d-a5c5-bf57cfe0a188 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:33.366239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:33.368546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:33.375023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f706c5-3e0b-4ed1-a23f-8c31ea6ff515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:33.375999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f706c5-3e0b-4ed1-a23f-8c31ea6ff515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:33.376348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f706c5-3e0b-4ed1-a23f-8c31ea6ff515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:33.382246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f706c5-3e0b-4ed1-a23f-8c31ea6ff515 admin admin] PATH_INFO: `/v3/projects/a96b5da347f744c9b305f206acb396fe/users/aaf859d57bfe47bdb28d8c44b2d52137/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:33.384516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f706c5-3e0b-4ed1-a23f-8c31ea6ff515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96b5da347f744c9b305f206acb396fe, user_id=aaf859d57bfe47bdb28d8c44b2d52137, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:33.387640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f706c5-3e0b-4ed1-a23f-8c31ea6ff515 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:33.389792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ec1b4-8604-49f1-baf2-2dfd495378bd admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:33.402980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:33.403987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:33.404407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:33.404976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] PATH_INFO: `/v3/users/ecd03c36e31e42588a5b87d3d3303aa9` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:33.406691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd03c36e31e42588a5b87d3d3303aa9)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:33.409124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:33.418493 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 694/1383] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:33 2022] PUT /identity/v3/projects/a96b5da347f744c9b305f206acb396fe/users/aaf859d57bfe47bdb28d8c44b2d52137/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:33.501634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:33.526496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd03c36e31e42588a5b87d3d3303aa9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:33.542999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd03c36e31e42588a5b87d3d3303aa9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:33.544686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd03c36e31e42588a5b87d3d3303aa9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:33.563305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd03c36e31e42588a5b87d3d3303aa9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:33.579374 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c9513f4e-edb6-403a-b0e5-4efa656988ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd03c36e31e42588a5b87d3d3303aa9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:33.596602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 690/1384] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:33 2022] DELETE /identity/v3/users/ecd03c36e31e42588a5b87d3d3303aa9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:33.601913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f09966-a2e1-439f-abf3-1ac50f7c28b0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:33.602448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f09966-a2e1-439f-abf3-1ac50f7c28b0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:33.602907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f09966-a2e1-439f-abf3-1ac50f7c28b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:33.645145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f0f09966-a2e1-439f-abf3-1ac50f7c28b0 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:33.649621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f0f09966-a2e1-439f-abf3-1ac50f7c28b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:33.746798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:33.834778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea75d-b81c-4f58-93dd-f91d2ca13cf1 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:33.835135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea75d-b81c-4f58-93dd-f91d2ca13cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:33.835591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea75d-b81c-4f58-93dd-f91d2ca13cf1 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:33.836971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 691/1385] 10.222.0.42 () {54 vars in 912 bytes} [Mon Oct 24 13:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:12:33.844984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419bd8d-2bbb-4694-ac4e-260907075a56 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:33.845275 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419bd8d-2bbb-4694-ac4e-260907075a56 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:33.845557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419bd8d-2bbb-4694-ac4e-260907075a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:33.852241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 695/1386] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:33.864648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f09966-a2e1-439f-abf3-1ac50f7c28b0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:33.867235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f0f09966-a2e1-439f-abf3-1ac50f7c28b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:33.954020 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7419bd8d-2bbb-4694-ac4e-260907075a56 None None] MFA Rules not processed for user `9c414b04fe0544e3a938661bc957c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:33.959466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7419bd8d-2bbb-4694-ac4e-260907075a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:34.029602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f09966-a2e1-439f-abf3-1ac50f7c28b0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:34.039487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3999c-4d60-4113-a2ea-edf9ff48143d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:34.042940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3999c-4d60-4113-a2ea-edf9ff48143d service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:34.043339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3999c-4d60-4113-a2ea-edf9ff48143d service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:34.044703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3999c-4d60-4113-a2ea-edf9ff48143d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:34.087883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 692/1387] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:12:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:34.090461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419bd8d-2bbb-4694-ac4e-260907075a56 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:34.110285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3999c-4d60-4113-a2ea-edf9ff48143d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:34.112167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3999c-4d60-4113-a2ea-edf9ff48143d service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:34.112411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7419bd8d-2bbb-4694-ac4e-260907075a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:34.139266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 696/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:34.143528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73137a2e-cc7c-493e-8bbd-630439422c1a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:34.144216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73137a2e-cc7c-493e-8bbd-630439422c1a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:34.146296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73137a2e-cc7c-493e-8bbd-630439422c1a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:34.148027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419bd8d-2bbb-4694-ac4e-260907075a56 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:34.155958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 697/1389] 10.222.0.42 () {54 vars in 912 bytes} [Mon Oct 24 13:12:34 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:12:34.157112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73137a2e-cc7c-493e-8bbd-630439422c1a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:34.166740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:34.169674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:34.170072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:34.170851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] PATH_INFO: `/v3/projects/6286671a05ed4748baec43c2c31bd089` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:34.175656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6286671a05ed4748baec43c2c31bd089)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:34.179577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:34.199625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73137a2e-cc7c-493e-8bbd-630439422c1a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:34.207576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3ab74-51b6-47f4-b2c4-39b8d276073c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:34.208458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3ab74-51b6-47f4-b2c4-39b8d276073c service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:34.209780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3ab74-51b6-47f4-b2c4-39b8d276073c service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:34.210653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3ab74-51b6-47f4-b2c4-39b8d276073c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:34.228573 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3ab74-51b6-47f4-b2c4-39b8d276073c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:34.232020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3ab74-51b6-47f4-b2c4-39b8d276073c service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:34.253999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 698/1390] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:34.255547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463551a-245c-4e83-9634-9830ebedfd4b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:34.255934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463551a-245c-4e83-9634-9830ebedfd4b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:34.256395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463551a-245c-4e83-9634-9830ebedfd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:34.306134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6286671a05ed4748baec43c2c31bd089', 'request_id': 'req-5658be49-8d06-4314-8110-5b618bbd6761'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.322359 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6286671a05ed4748baec43c2c31bd089', 'request_id': 'req-5658be49-8d06-4314-8110-5b618bbd6761'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.332350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a463551a-245c-4e83-9634-9830ebedfd4b None None] MFA Rules not processed for user `3e02f6f701324af6b53762569f1a9d21`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:34.333949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a463551a-245c-4e83-9634-9830ebedfd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:34.342157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6286671a05ed4748baec43c2c31bd089', 'request_id': 'req-5658be49-8d06-4314-8110-5b618bbd6761'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.362334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] The token cache is being invalidate because project 6286671a05ed4748baec43c2c31bd089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:34.363225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.364928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5658be49-8d06-4314-8110-5b618bbd6761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.368356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 693/1391] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:34 2022] DELETE /identity/v3/projects/6286671a05ed4748baec43c2c31bd089 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:34.373329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67280d16-e033-4035-8807-ca360f137128 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:34.376112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67280d16-e033-4035-8807-ca360f137128 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:34.376513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67280d16-e033-4035-8807-ca360f137128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:34.436374 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-67280d16-e033-4035-8807-ca360f137128 None None] MFA Rules not processed for user `9c414b04fe0544e3a938661bc957c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:34.444571 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-67280d16-e033-4035-8807-ca360f137128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:34.485005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:34.525385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 699/1392] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:34.533915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463551a-245c-4e83-9634-9830ebedfd4b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:34.533915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a463551a-245c-4e83-9634-9830ebedfd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:34.570664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 694/1393] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:34.660722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67280d16-e033-4035-8807-ca360f137128 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:34.664289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-67280d16-e033-4035-8807-ca360f137128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:34.669363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463551a-245c-4e83-9634-9830ebedfd4b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:34.678178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:34.679114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:34.679408 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:34.679683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] PATH_INFO: `/v3/users/71896fc97c304b2e8454b2053c2f86ea` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:34.680573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67280d16-e033-4035-8807-ca360f137128 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:34.681077 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71896fc97c304b2e8454b2053c2f86ea)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:34.684374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:34.687210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:34.691801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:34.693966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:34.695837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:34.697373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:34.720378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:34.724229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:34.746018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 695/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:12:34.757924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71896fc97c304b2e8454b2053c2f86ea'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.782346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71896fc97c304b2e8454b2053c2f86ea'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.783329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71896fc97c304b2e8454b2053c2f86ea'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.795357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71896fc97c304b2e8454b2053c2f86ea'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.799331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71896fc97c304b2e8454b2053c2f86ea'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:34.818101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 700/1395] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:34 2022] DELETE /identity/v3/users/71896fc97c304b2e8454b2053c2f86ea => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:35.324607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:35.326581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:35.366327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fdfff0-e2ac-4982-ad59-4185a6743c7b service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:35.393433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:35.395645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:35.537659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:35.539433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:35.539915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:35.540889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] PATH_INFO: `/v3/projects/1db6666d568b466ea2fa1d3c450fd380` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:35.542762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1db6666d568b466ea2fa1d3c450fd380)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:35.545642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:35.623524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e4482c-e4ed-4bd5-abac-6c8e25131c11 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:35.642770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1db6666d568b466ea2fa1d3c450fd380', 'request_id': 'req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.663039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1db6666d568b466ea2fa1d3c450fd380', 'request_id': 'req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.681782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1db6666d568b466ea2fa1d3c450fd380', 'request_id': 'req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.716096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] The token cache is being invalidate because project 1db6666d568b466ea2fa1d3c450fd380 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:35.717064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.721299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.724800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 696/1396] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:35 2022] DELETE /identity/v3/projects/1db6666d568b466ea2fa1d3c450fd380 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:35.727987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:35.732566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:35.783802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:35.784590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:35.784966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:35.785316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] PATH_INFO: `/v3/users/e5bcdd36246d4731a7e2f43e1bd63a65` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:35.786551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5bcdd36246d4731a7e2f43e1bd63a65)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:35.788623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:35.854068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:35.864128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:35.913190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5bcdd36246d4731a7e2f43e1bd63a65'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.925137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b34627-f1d3-4d31-9b6e-32a35a7b19f4 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:35.942286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:35.943099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:35.943465 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:35.943808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:35.945448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5bcdd36246d4731a7e2f43e1bd63a65'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.946028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5bcdd36246d4731a7e2f43e1bd63a65'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.947232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:35.963926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5bcdd36246d4731a7e2f43e1bd63a65'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.964572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5bcdd36246d4731a7e2f43e1bd63a65'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:35.985848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 701/1397] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:35 2022] DELETE /identity/v3/users/e5bcdd36246d4731a7e2f43e1bd63a65 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:35.986841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:35.991242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:36.096385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:36.098369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:36.160031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 697/1398] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:36.160031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:36.161166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:36.175660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2850208-b544-40ef-b6e1-a3c0835a33e4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:36.190756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61735360-bb77-4d3c-9164-ff611058e81b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:36.191977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61735360-bb77-4d3c-9164-ff611058e81b service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:36.193974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61735360-bb77-4d3c-9164-ff611058e81b service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:36.193974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61735360-bb77-4d3c-9164-ff611058e81b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:36.195162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d9c14-b484-4736-b956-4866edf9cdf0 service nova] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:36.196365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-61735360-bb77-4d3c-9164-ff611058e81b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:36.244602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61735360-bb77-4d3c-9164-ff611058e81b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:36.258599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61735360-bb77-4d3c-9164-ff611058e81b service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:36.312462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 702/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:36.317444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e2e9-eb6f-425d-8ab7-1fb3e2a3e993 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:36.317824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e2e9-eb6f-425d-8ab7-1fb3e2a3e993 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:36.318201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725e2e9-eb6f-425d-8ab7-1fb3e2a3e993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:36.441019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:36.441837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:36.442216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:36.442587 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] PATH_INFO: `/v3/users/edcc51c2df9e4c0aa71581b0c2f74f80` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:36.447978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edcc51c2df9e4c0aa71581b0c2f74f80)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:36.450179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f725e2e9-eb6f-425d-8ab7-1fb3e2a3e993 None None] MFA Rules not processed for user `aaf859d57bfe47bdb28d8c44b2d52137`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:36.450584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:36.459954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f725e2e9-eb6f-425d-8ab7-1fb3e2a3e993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:36.586554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edcc51c2df9e4c0aa71581b0c2f74f80'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:36.630498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edcc51c2df9e4c0aa71581b0c2f74f80'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:36.633141 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edcc51c2df9e4c0aa71581b0c2f74f80'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:36.670791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edcc51c2df9e4c0aa71581b0c2f74f80'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:36.696076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edcc51c2df9e4c0aa71581b0c2f74f80'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:36.721826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 698/1400] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:36 2022] DELETE /identity/v3/users/edcc51c2df9e4c0aa71581b0c2f74f80 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:36.722853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:36.741069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:36.795463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:36.857656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 703/1401] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:36 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 540 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:36.857656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33da7a-47dc-47d2-ab50-fc97b3967ad2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:36.857656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33da7a-47dc-47d2-ab50-fc97b3967ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:36.857656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33da7a-47dc-47d2-ab50-fc97b3967ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:36.907892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3c33da7a-47dc-47d2-ab50-fc97b3967ad2 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:36.909384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3c33da7a-47dc-47d2-ab50-fc97b3967ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:36.940150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:36.994054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba4f32-df6d-4f34-8574-898ae7100997 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:37.007912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:37.009997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:37.010351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:37.011759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] PATH_INFO: `/v3/projects/3b08d3a7d9be417dab442e1b9363c751` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:37.016033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b08d3a7d9be417dab442e1b9363c751)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:37.019547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:37.064274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 704/1402] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:37.065369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33da7a-47dc-47d2-ab50-fc97b3967ad2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:37.067343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3c33da7a-47dc-47d2-ab50-fc97b3967ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:37.153870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b08d3a7d9be417dab442e1b9363c751', 'request_id': 'req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:37.173584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b08d3a7d9be417dab442e1b9363c751', 'request_id': 'req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:37.190388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b08d3a7d9be417dab442e1b9363c751', 'request_id': 'req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:37.235097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] The token cache is being invalidate because project 3b08d3a7d9be417dab442e1b9363c751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:37.235395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:37.239852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2bc3bcad-15b7-4c37-9e27-e59cc31ee8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:37.242155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 699/1403] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:36 2022] DELETE /identity/v3/projects/3b08d3a7d9be417dab442e1b9363c751 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:37.253155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396390e-0d6b-487f-a06b-39bc64f40901 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:37.253155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396390e-0d6b-487f-a06b-39bc64f40901 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:37.253155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396390e-0d6b-487f-a06b-39bc64f40901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:37.255226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33da7a-47dc-47d2-ab50-fc97b3967ad2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:37.336679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-2396390e-0d6b-487f-a06b-39bc64f40901 None None] MFA Rules not processed for user `d29ff25b062346f6bcf9910b1fa8db5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:37.339734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2396390e-0d6b-487f-a06b-39bc64f40901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:37.452346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef3edb-9d06-42cb-b6e6-6a3152f66597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:37.453036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef3edb-9d06-42cb-b6e6-6a3152f66597 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:37.453321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef3edb-9d06-42cb-b6e6-6a3152f66597 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:37.454331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef3edb-9d06-42cb-b6e6-6a3152f66597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:37.456112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f0ef3edb-9d06-42cb-b6e6-6a3152f66597 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:37.487742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 700/1404] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:37.487742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396390e-0d6b-487f-a06b-39bc64f40901 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:37.490746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2396390e-0d6b-487f-a06b-39bc64f40901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:37.588322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:37.591818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef3edb-9d06-42cb-b6e6-6a3152f66597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:37.594161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef3edb-9d06-42cb-b6e6-6a3152f66597 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:37.620400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 705/1405] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:37.622580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ef368-ceda-4b46-9f2f-f36a97005acf None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:37.623111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ef368-ceda-4b46-9f2f-f36a97005acf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:37.623990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ef368-ceda-4b46-9f2f-f36a97005acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:37.630308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:37.656735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396390e-0d6b-487f-a06b-39bc64f40901 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:37.665081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da7139-c4ad-4e52-a8e5-df1e070dbc26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:37.665718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da7139-c4ad-4e52-a8e5-df1e070dbc26 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:37.665999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da7139-c4ad-4e52-a8e5-df1e070dbc26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:37.666274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da7139-c4ad-4e52-a8e5-df1e070dbc26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:37.673576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b6da7139-c4ad-4e52-a8e5-df1e070dbc26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:37.680406 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-906ef368-ceda-4b46-9f2f-f36a97005acf None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:37.684570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-906ef368-ceda-4b46-9f2f-f36a97005acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:37.909837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da7139-c4ad-4e52-a8e5-df1e070dbc26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:37.919800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da7139-c4ad-4e52-a8e5-df1e070dbc26 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:37.951520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 701/1406] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:38.018062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 706/1407] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:38.024948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb5779-2a97-41d4-bc83-79486bd41665 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:38.025241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb5779-2a97-41d4-bc83-79486bd41665 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:38.025545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb5779-2a97-41d4-bc83-79486bd41665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:38.095489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-19fb5779-2a97-41d4-bc83-79486bd41665 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:38.096739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-19fb5779-2a97-41d4-bc83-79486bd41665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:38.140628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 702/1408] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:38.141794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ef368-ceda-4b46-9f2f-f36a97005acf None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:38.168304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ef368-ceda-4b46-9f2f-f36a97005acf None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:38.184837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:38.184837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:38.184837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:38.184837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:38.188559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:38.189193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:38.194718 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:38.223737 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 707/1409] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:12:38.228865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb5779-2a97-41d4-bc83-79486bd41665 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:38.246834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb5779-2a97-41d4-bc83-79486bd41665 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:38.285183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:38.286114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:38.286680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:38.287194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:38.287968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:38.294851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:38.376690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 703/1410] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:38 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:38.377724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:38.417490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ce4a2-d9e7-498d-8e82-da83d732315e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:38.481213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:38.483044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:38.624531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6086a5-bda4-443d-8254-4d851f9fc5c7 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:38.633128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e4789-0045-429e-8f8d-6c289d91e265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:38.633934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4789-0045-429e-8f8d-6c289d91e265 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:38.634414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4789-0045-429e-8f8d-6c289d91e265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:38.635523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e4789-0045-429e-8f8d-6c289d91e265 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:38.637236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e4789-0045-429e-8f8d-6c289d91e265 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:38.643033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e4789-0045-429e-8f8d-6c289d91e265 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:38.646480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:38.649020 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:38.649568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:38.650259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] PATH_INFO: `/v3/projects/770206629c1d4eb89e5ddf603e7708eb` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:38.652021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=770206629c1d4eb89e5ddf603e7708eb)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:38.654684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:38.699254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 708/1411] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:38 2022] POST /identity/v3/users => generated 291 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:38.700884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e4789-0045-429e-8f8d-6c289d91e265 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:38.730157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e4789-0045-429e-8f8d-6c289d91e265 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:38.743290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5e03-8ec5-4511-bb3c-98ef182161f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:38.744185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5e03-8ec5-4511-bb3c-98ef182161f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:38.744587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5e03-8ec5-4511-bb3c-98ef182161f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:38.744978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a5e03-8ec5-4511-bb3c-98ef182161f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:38.745841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a5e03-8ec5-4511-bb3c-98ef182161f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:38.747660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a5e03-8ec5-4511-bb3c-98ef182161f3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:38.759919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '770206629c1d4eb89e5ddf603e7708eb', 'request_id': 'req-854b5b52-a53e-4052-85cf-15599f2fa635'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:38.775914 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 709/1412] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:38.788207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5e03-8ec5-4511-bb3c-98ef182161f3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:38.789630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '770206629c1d4eb89e5ddf603e7708eb', 'request_id': 'req-854b5b52-a53e-4052-85cf-15599f2fa635'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:38.817777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '770206629c1d4eb89e5ddf603e7708eb', 'request_id': 'req-854b5b52-a53e-4052-85cf-15599f2fa635'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:38.832977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a5e03-8ec5-4511-bb3c-98ef182161f3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:38.834877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:38.865829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] The token cache is being invalidate because project 770206629c1d4eb89e5ddf603e7708eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:38.866586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:38.868609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-854b5b52-a53e-4052-85cf-15599f2fa635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:38.871578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 704/1413] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:38 2022] DELETE /identity/v3/projects/770206629c1d4eb89e5ddf603e7708eb => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:38.882954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010853f0-0f51-426f-b621-d9593f476111 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:38.884246 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010853f0-0f51-426f-b621-d9593f476111 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:38.885656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010853f0-0f51-426f-b621-d9593f476111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:38.982896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-010853f0-0f51-426f-b621-d9593f476111 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:38.986622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-010853f0-0f51-426f-b621-d9593f476111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:39.043859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:39.047120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:39.047749 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:39.048239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:39.049188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:39.057284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:39.060971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:39.086424 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 710/1414] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:12:39.091105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:39.096987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:39.117863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f140e1-b873-4080-b188-1f206fbc2122 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:39.123255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c0b42-93dc-495d-98a2-ce1d4b4b50e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:39.124285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c0b42-93dc-495d-98a2-ce1d4b4b50e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:39.124752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c0b42-93dc-495d-98a2-ce1d4b4b50e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:39.125138 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c0b42-93dc-495d-98a2-ce1d4b4b50e3 admin admin] PATH_INFO: `/v3/projects/07e225871dfc4e7587af04d7ca61f24b/users/b3a9d19d155946b4bd0021ac54e62a3f/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:39.191525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c0b42-93dc-495d-98a2-ce1d4b4b50e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07e225871dfc4e7587af04d7ca61f24b, user_id=b3a9d19d155946b4bd0021ac54e62a3f, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:39.201132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c0b42-93dc-495d-98a2-ce1d4b4b50e3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:39.229961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 705/1415] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:39.246636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 711/1416] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:39 2022] PUT /identity/v3/projects/07e225871dfc4e7587af04d7ca61f24b/users/b3a9d19d155946b4bd0021ac54e62a3f/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:39.247384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f4a53-74a9-4141-a26d-2a3c1471b59e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:39.247384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f4a53-74a9-4141-a26d-2a3c1471b59e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:39.247384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f4a53-74a9-4141-a26d-2a3c1471b59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:39.251485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d987-8fb2-4d3f-b964-842738555816 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:39.251937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d987-8fb2-4d3f-b964-842738555816 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:39.253586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026d987-8fb2-4d3f-b964-842738555816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:39.325246 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-040f4a53-74a9-4141-a26d-2a3c1471b59e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:39.329865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-040f4a53-74a9-4141-a26d-2a3c1471b59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:39.345130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c026d987-8fb2-4d3f-b964-842738555816 None None] MFA Rules not processed for user `b3a9d19d155946b4bd0021ac54e62a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:39.348554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c026d987-8fb2-4d3f-b964-842738555816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:39.462512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 706/1417] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:39.463593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f4a53-74a9-4141-a26d-2a3c1471b59e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:39.465022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-040f4a53-74a9-4141-a26d-2a3c1471b59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:39.501415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f4a53-74a9-4141-a26d-2a3c1471b59e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:39.522531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 712/1418] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:39.524738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026d987-8fb2-4d3f-b964-842738555816 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:39.527101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:39.527976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:39.530433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:39.531140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] PATH_INFO: `/v3/projects/d05ad5008e7c411da8d8f5d4762e55b1` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:39.540277 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05ad5008e7c411da8d8f5d4762e55b1)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:39.549649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:39.560657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026d987-8fb2-4d3f-b964-842738555816 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:39.571923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:39.572817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:39.573247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:39.573667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:39.574475 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:39.575449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:39.579422 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:39.643292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 713/1419] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:39.644622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:39.678897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281d824-b6a3-4322-a5d2-b8db7c41f39c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:39.704404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05ad5008e7c411da8d8f5d4762e55b1', 'request_id': 'req-a233c244-d19d-4b6a-84b0-e21980b72976'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:39.720011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05ad5008e7c411da8d8f5d4762e55b1', 'request_id': 'req-a233c244-d19d-4b6a-84b0-e21980b72976'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:39.753351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05ad5008e7c411da8d8f5d4762e55b1', 'request_id': 'req-a233c244-d19d-4b6a-84b0-e21980b72976'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:39.794925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] The token cache is being invalidate because project d05ad5008e7c411da8d8f5d4762e55b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:39.795862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:39.799802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:39.809438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 707/1420] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:39 2022] DELETE /identity/v3/projects/d05ad5008e7c411da8d8f5d4762e55b1 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:39.817809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:39.822196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:39.889853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:39.890505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:39.895373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:39.895680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:39.896356 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:39.904464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:39.946303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:39.964028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 714/1421] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:39 2022] POST /identity/v3/projects => generated 379 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:39.967042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:39.975089 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:40.099094 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:40.162796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01836302-d81d-4d1f-a249-eb92edc4ca20 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:40.176160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233c244-d19d-4b6a-84b0-e21980b72976 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:40.179707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:40.180384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.180851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.180961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] PATH_INFO: `/v3/users/f6d1f5beb6404090a8aaca0ff54a968a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.182395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6d1f5beb6404090a8aaca0ff54a968a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:40.184450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:40.196013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:40.198323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.199492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.200556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.202021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:40.207847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:40.284120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 708/1422] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:39 2022] POST /identity/v3/projects => generated 363 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:40.285149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:40.287147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:40.306373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6d1f5beb6404090a8aaca0ff54a968a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:40.336964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6d1f5beb6404090a8aaca0ff54a968a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:40.339144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6d1f5beb6404090a8aaca0ff54a968a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:40.364762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6d1f5beb6404090a8aaca0ff54a968a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:40.365602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6d1f5beb6404090a8aaca0ff54a968a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:40.388413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 715/1423] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:39 2022] DELETE /identity/v3/users/f6d1f5beb6404090a8aaca0ff54a968a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:40.396128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:40.402165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:40.454092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bdd919-f7a5-4f41-a8ab-3b71b2fbf7e5 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:40.571666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3430300e-c053-42e6-9486-b21775d5dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:40.572861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430300e-c053-42e6-9486-b21775d5dd25 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.573335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430300e-c053-42e6-9486-b21775d5dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.573752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3430300e-c053-42e6-9486-b21775d5dd25 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.574579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430300e-c053-42e6-9486-b21775d5dd25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:40.577059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3430300e-c053-42e6-9486-b21775d5dd25 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:40.607914 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593edab7-4f05-4613-ad74-157c75ebe869 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:40.608838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 709/1424] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:40 2022] POST /identity/v3/users => generated 296 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:40.611017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5744-da6f-48b1-ba8f-bb72331cf0c9 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.611467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5744-da6f-48b1-ba8f-bb72331cf0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.612662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5744-da6f-48b1-ba8f-bb72331cf0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.620587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:40.621243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.621530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.621812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.623161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:40.630847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:40.674990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 716/1425] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:40 2022] POST /identity/v3/users => generated 282 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:40.677853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:40.683085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:40.720232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41b67d-3ff9-40ca-ad21-1dfe989d0966 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:40.737796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c01c3-4a46-4ba6-908c-6e61ef0685ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:40.738620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c01c3-4a46-4ba6-908c-6e61ef0685ca admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.738969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c01c3-4a46-4ba6-908c-6e61ef0685ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.739503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c01c3-4a46-4ba6-908c-6e61ef0685ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.741377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c01c3-4a46-4ba6-908c-6e61ef0685ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:40.748543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c01c3-4a46-4ba6-908c-6e61ef0685ca admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:40.751632 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-afaf5744-da6f-48b1-ba8f-bb72331cf0c9 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:40.756816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-afaf5744-da6f-48b1-ba8f-bb72331cf0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:40.771802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 717/1426] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:40.777530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:40.781040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c01c3-4a46-4ba6-908c-6e61ef0685ca admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:40.813934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c01c3-4a46-4ba6-908c-6e61ef0685ca admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:40.823610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4beae-e32b-4fb2-8ab0-d058b7e6aabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:40.824516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4beae-e32b-4fb2-8ab0-d058b7e6aabd admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.824808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4beae-e32b-4fb2-8ab0-d058b7e6aabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.825083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4beae-e32b-4fb2-8ab0-d058b7e6aabd admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.825748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4beae-e32b-4fb2-8ab0-d058b7e6aabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:40.828110 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4beae-e32b-4fb2-8ab0-d058b7e6aabd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:40.898642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 718/1427] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:40.901768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4beae-e32b-4fb2-8ab0-d058b7e6aabd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:40.910440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4beae-e32b-4fb2-8ab0-d058b7e6aabd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:40.918379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747d1fe-094e-49f1-9249-a7104daaed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:40.919283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747d1fe-094e-49f1-9249-a7104daaed9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.919490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747d1fe-094e-49f1-9249-a7104daaed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.919779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747d1fe-094e-49f1-9249-a7104daaed9f admin admin] PATH_INFO: `/v3/projects/391ed02c7029442f8acedd097b32bc76/users/01f050da05ae47ae8731bb1cd4abfdd7/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.946425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 710/1428] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:12:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:40.947410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf5744-da6f-48b1-ba8f-bb72331cf0c9 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:40.961965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747d1fe-094e-49f1-9249-a7104daaed9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=391ed02c7029442f8acedd097b32bc76, user_id=01f050da05ae47ae8731bb1cd4abfdd7, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:40.967297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747d1fe-094e-49f1-9249-a7104daaed9f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:40.968965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf5744-da6f-48b1-ba8f-bb72331cf0c9 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:40.977946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc01d1-c581-4974-a13c-a1a46e0c7b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:40.978829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc01d1-c581-4974-a13c-a1a46e0c7b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:40.979243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc01d1-c581-4974-a13c-a1a46e0c7b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:40.979526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc01d1-c581-4974-a13c-a1a46e0c7b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:40.980296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc01d1-c581-4974-a13c-a1a46e0c7b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:40.983337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc01d1-c581-4974-a13c-a1a46e0c7b3a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:40.992591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:40.998066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 711/1429] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:41.005730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc01d1-c581-4974-a13c-a1a46e0c7b3a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:41.020376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc01d1-c581-4974-a13c-a1a46e0c7b3a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:41.023308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 719/1430] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:40 2022] PUT /identity/v3/projects/391ed02c7029442f8acedd097b32bc76/users/01f050da05ae47ae8731bb1cd4abfdd7/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:41.027276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747d1fe-094e-49f1-9249-a7104daaed9f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:41.211269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314fa1b-7634-4fd4-a447-849459e5f868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:41.211962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314fa1b-7634-4fd4-a447-849459e5f868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:41.212243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314fa1b-7634-4fd4-a447-849459e5f868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:41.213105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314fa1b-7634-4fd4-a447-849459e5f868 admin admin] PATH_INFO: `/v3/projects/fecdd1980bcf4f9ea66d7e0724b868d0/users/0d7515577a5b49d99d347c3c8c401b7a/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:41.225651 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747d1fe-094e-49f1-9249-a7104daaed9f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:41.237543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d369aaf-232b-4414-8b67-75f62822aa4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:41.238389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d369aaf-232b-4414-8b67-75f62822aa4f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:41.239729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d369aaf-232b-4414-8b67-75f62822aa4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:41.240161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d369aaf-232b-4414-8b67-75f62822aa4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:41.241290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d369aaf-232b-4414-8b67-75f62822aa4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:41.243392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d369aaf-232b-4414-8b67-75f62822aa4f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:41.243863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314fa1b-7634-4fd4-a447-849459e5f868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecdd1980bcf4f9ea66d7e0724b868d0, user_id=0d7515577a5b49d99d347c3c8c401b7a, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:41.262423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 720/1431] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:41.263853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a06ad6-c1d0-4f50-845c-dd66df4e42c3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:41.264207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a06ad6-c1d0-4f50-845c-dd66df4e42c3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:41.264487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a06ad6-c1d0-4f50-845c-dd66df4e42c3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:41.266947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 721/1432] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:12:41.267932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a06ad6-c1d0-4f50-845c-dd66df4e42c3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:41.268062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314fa1b-7634-4fd4-a447-849459e5f868 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:41.269765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-35a06ad6-c1d0-4f50-845c-dd66df4e42c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:41.319536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 712/1433] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:41 2022] PUT /identity/v3/projects/fecdd1980bcf4f9ea66d7e0724b868d0/users/0d7515577a5b49d99d347c3c8c401b7a/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:12:41.320091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314fa1b-7634-4fd4-a447-849459e5f868 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:41.527382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a06ad6-c1d0-4f50-845c-dd66df4e42c3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:41.560013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314fa1b-7634-4fd4-a447-849459e5f868 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:41.574473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac72ba8-7ce0-4c83-9569-eef647e49bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:41.575200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac72ba8-7ce0-4c83-9569-eef647e49bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:41.575555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac72ba8-7ce0-4c83-9569-eef647e49bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:41.575902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac72ba8-7ce0-4c83-9569-eef647e49bd2 admin admin] PATH_INFO: `/v3/projects/391ed02c7029442f8acedd097b32bc76/users/01f050da05ae47ae8731bb1cd4abfdd7/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:41.584156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac72ba8-7ce0-4c83-9569-eef647e49bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391ed02c7029442f8acedd097b32bc76, user_id=01f050da05ae47ae8731bb1cd4abfdd7, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:41.587002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac72ba8-7ce0-4c83-9569-eef647e49bd2 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:41.599301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 713/1434] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:41 2022] PUT /identity/v3/projects/391ed02c7029442f8acedd097b32bc76/users/01f050da05ae47ae8731bb1cd4abfdd7/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:41.600696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdaa6-4578-4005-9e66-8cc5300aaf03 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:41.600984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdaa6-4578-4005-9e66-8cc5300aaf03 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:41.601318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cdaa6-4578-4005-9e66-8cc5300aaf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:41.627859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-fa2cdaa6-4578-4005-9e66-8cc5300aaf03 None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:41.629342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fa2cdaa6-4578-4005-9e66-8cc5300aaf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:41.637886 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88517c0b-1f51-4bb3-a253-ec8ff4a51b00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:41.638789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88517c0b-1f51-4bb3-a253-ec8ff4a51b00 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:41.642038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88517c0b-1f51-4bb3-a253-ec8ff4a51b00 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:41.642405 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88517c0b-1f51-4bb3-a253-ec8ff4a51b00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:41.652116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:41.699623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:41.777604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88517c0b-1f51-4bb3-a253-ec8ff4a51b00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:41.778125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 714/1435] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:12:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:41.779250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88517c0b-1f51-4bb3-a253-ec8ff4a51b00 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:41.786912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850153f-3f9f-45ac-ad08-507144cf7761 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:41.787299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850153f-3f9f-45ac-ad08-507144cf7761 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:41.787588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d850153f-3f9f-45ac-ad08-507144cf7761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:41.806805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 722/1436] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:12:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:12:41.808519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24471de5-c425-4b76-97d9-45ddaca792a2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:41.809038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24471de5-c425-4b76-97d9-45ddaca792a2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:41.809512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24471de5-c425-4b76-97d9-45ddaca792a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:41.867030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d850153f-3f9f-45ac-ad08-507144cf7761 None None] MFA Rules not processed for user `0d7515577a5b49d99d347c3c8c401b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:41.872218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d850153f-3f9f-45ac-ad08-507144cf7761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:41.878164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-24471de5-c425-4b76-97d9-45ddaca792a2 None None] MFA Rules not processed for user `01f050da05ae47ae8731bb1cd4abfdd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:41.880136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-24471de5-c425-4b76-97d9-45ddaca792a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:42.103862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 723/1437] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:42.106842 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3017f-0e9e-447b-a7e6-fbabbc490641 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:42.107188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3017f-0e9e-447b-a7e6-fbabbc490641 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:42.107470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f3017f-0e9e-447b-a7e6-fbabbc490641 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:42.119664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 724/1438] 10.222.0.42 () {54 vars in 961 bytes} [Mon Oct 24 13:12:42 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:12:42.122266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3017f-0e9e-447b-a7e6-fbabbc490641 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:42.128301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-82f3017f-0e9e-447b-a7e6-fbabbc490641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:42.220858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 715/1439] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:42.226602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5313-fe88-45b8-81dd-998ed2394b5b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:42.226954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5313-fe88-45b8-81dd-998ed2394b5b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:42.227269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5313-fe88-45b8-81dd-998ed2394b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:42.310121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-fa0c5313-fe88-45b8-81dd-998ed2394b5b None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:42.312075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fa0c5313-fe88-45b8-81dd-998ed2394b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:42.367676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f3017f-0e9e-447b-a7e6-fbabbc490641 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:42.369076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 716/1440] 10.222.0.42 () {58 vars in 1058 bytes} [Mon Oct 24 13:12:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:42.371042 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c5313-fe88-45b8-81dd-998ed2394b5b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:42.377511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a795cc9-6655-4645-9a8d-51447a9e1aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:42.378444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a795cc9-6655-4645-9a8d-51447a9e1aaa service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:42.378866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a795cc9-6655-4645-9a8d-51447a9e1aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:42.380099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a795cc9-6655-4645-9a8d-51447a9e1aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:42.397771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a795cc9-6655-4645-9a8d-51447a9e1aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:42.400333 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a795cc9-6655-4645-9a8d-51447a9e1aaa service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:42.445276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 725/1441] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:42 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:42.447166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a795cc9-6655-4645-9a8d-51447a9e1aaa service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:42.512195 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a795cc9-6655-4645-9a8d-51447a9e1aaa service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:42.529548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3edb2-02b6-4b28-97a2-e8be906de788 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:42.531248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3edb2-02b6-4b28-97a2-e8be906de788 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:42.531680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3edb2-02b6-4b28-97a2-e8be906de788 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:42.533031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3edb2-02b6-4b28-97a2-e8be906de788 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:42.558333 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3edb2-02b6-4b28-97a2-e8be906de788 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:42.562410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3edb2-02b6-4b28-97a2-e8be906de788 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:42.581256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c5313-fe88-45b8-81dd-998ed2394b5b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:42.591048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 726/1442] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:42.629581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bede12c-512f-49fb-abe6-cbbd21b15177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:42.630245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bede12c-512f-49fb-abe6-cbbd21b15177 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:42.630643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bede12c-512f-49fb-abe6-cbbd21b15177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:42.630885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bede12c-512f-49fb-abe6-cbbd21b15177 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:42.631569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bede12c-512f-49fb-abe6-cbbd21b15177 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:42.638722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bede12c-512f-49fb-abe6-cbbd21b15177 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:42.748744 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 717/1443] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:42 2022] POST /identity/v3/projects => generated 363 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:42.756178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3edb2-02b6-4b28-97a2-e8be906de788 service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:42.781679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3edb2-02b6-4b28-97a2-e8be906de788 service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:42.889394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:42.939514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a165343-9ec1-4e45-b725-44a8725169dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:42.940660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a165343-9ec1-4e45-b725-44a8725169dc admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:42.941067 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a165343-9ec1-4e45-b725-44a8725169dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:42.941520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a165343-9ec1-4e45-b725-44a8725169dc admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:42.942261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a165343-9ec1-4e45-b725-44a8725169dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:42.945428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a165343-9ec1-4e45-b725-44a8725169dc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:43.007707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 727/1444] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:42 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:43.009064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bede12c-512f-49fb-abe6-cbbd21b15177 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:43.025050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bede12c-512f-49fb-abe6-cbbd21b15177 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:43.035250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41709263-23ce-4d53-bf33-f0294b79dc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:43.036673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41709263-23ce-4d53-bf33-f0294b79dc6a admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:43.037338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41709263-23ce-4d53-bf33-f0294b79dc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:43.037719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41709263-23ce-4d53-bf33-f0294b79dc6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:43.038489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41709263-23ce-4d53-bf33-f0294b79dc6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:43.041351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41709263-23ce-4d53-bf33-f0294b79dc6a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:43.079092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 718/1445] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:43.079277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a165343-9ec1-4e45-b725-44a8725169dc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:43.095139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a165343-9ec1-4e45-b725-44a8725169dc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:43.122296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0faef-903b-40bd-8355-f9630b07b6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:43.123005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0faef-903b-40bd-8355-f9630b07b6af admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:43.123315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0faef-903b-40bd-8355-f9630b07b6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:43.123599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0faef-903b-40bd-8355-f9630b07b6af admin admin] PATH_INFO: `/v3/projects/3060d830aa424d0284f2c59eb096ae55/users/51b87de9e37e44f59ba9888d70ceb8eb/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:43.178256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0faef-903b-40bd-8355-f9630b07b6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=3060d830aa424d0284f2c59eb096ae55, user_id=51b87de9e37e44f59ba9888d70ceb8eb, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:43.181006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0faef-903b-40bd-8355-f9630b07b6af admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:43.229390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 728/1446] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:43 2022] PUT /identity/v3/projects/3060d830aa424d0284f2c59eb096ae55/users/51b87de9e37e44f59ba9888d70ceb8eb/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:43.234830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41709263-23ce-4d53-bf33-f0294b79dc6a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:43.260907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41709263-23ce-4d53-bf33-f0294b79dc6a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:43.284580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:43.424086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:43.424511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:43.424797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:43.425078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:43.427339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:43.444902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:43.512974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 719/1447] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:43.517955 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0faef-903b-40bd-8355-f9630b07b6af admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:43.530838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0faef-903b-40bd-8355-f9630b07b6af admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:43.543304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f46201-6821-4f9e-aa22-72b55bb508b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:43.544029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f46201-6821-4f9e-aa22-72b55bb508b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:43.544874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f46201-6821-4f9e-aa22-72b55bb508b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:43.545175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f46201-6821-4f9e-aa22-72b55bb508b9 admin admin] PATH_INFO: `/v3/projects/3060d830aa424d0284f2c59eb096ae55/users/51b87de9e37e44f59ba9888d70ceb8eb/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:43.551191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f46201-6821-4f9e-aa22-72b55bb508b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3060d830aa424d0284f2c59eb096ae55, user_id=51b87de9e37e44f59ba9888d70ceb8eb, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:43.570090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f46201-6821-4f9e-aa22-72b55bb508b9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:43.583311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:43.590242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:43.616425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60f5cf-5642-41da-8387-570df5dc78e3 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:43.619150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 729/1448] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:43 2022] PUT /identity/v3/projects/3060d830aa424d0284f2c59eb096ae55/users/51b87de9e37e44f59ba9888d70ceb8eb/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:43.631799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8acea-0da5-4d74-9702-b0d8fc0bed96 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:43.634931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8acea-0da5-4d74-9702-b0d8fc0bed96 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:43.635540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8acea-0da5-4d74-9702-b0d8fc0bed96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:43.716145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f9e8acea-0da5-4d74-9702-b0d8fc0bed96 None None] MFA Rules not processed for user `51b87de9e37e44f59ba9888d70ceb8eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:43.718612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f9e8acea-0da5-4d74-9702-b0d8fc0bed96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:43.782092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:43.782768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:43.783086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:43.783355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] PATH_INFO: `/v3/projects/115397ad2fbf44d7a8316ef82509d4dd` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:43.784774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=115397ad2fbf44d7a8316ef82509d4dd)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:43.786856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:43.947524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 730/1449] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:43.948652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8acea-0da5-4d74-9702-b0d8fc0bed96 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:43.970035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115397ad2fbf44d7a8316ef82509d4dd', 'request_id': 'req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:43.972565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8acea-0da5-4d74-9702-b0d8fc0bed96 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:43.991910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115397ad2fbf44d7a8316ef82509d4dd', 'request_id': 'req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.004115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115397ad2fbf44d7a8316ef82509d4dd', 'request_id': 'req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.029553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] The token cache is being invalidate because project 115397ad2fbf44d7a8316ef82509d4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:44.030633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.032660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.038838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 720/1450] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:43 2022] DELETE /identity/v3/projects/115397ad2fbf44d7a8316ef82509d4dd => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:44.040158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:44.043971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:44.135972 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:44.136904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:44.137373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:44.137659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] PATH_INFO: `/v3/users/b3a9d19d155946b4bd0021ac54e62a3f` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:44.139369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a9d19d155946b4bd0021ac54e62a3f)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:44.143534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:44.145968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:44.173582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9bd2a-5966-413a-969c-5d7e5d6cdbb8 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:44.178153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abe32eb-141a-4376-b1aa-0725ef4e681a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:44.179081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe32eb-141a-4376-b1aa-0725ef4e681a service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:44.179547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:44.179831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe32eb-141a-4376-b1aa-0725ef4e681a service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:44.180264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abe32eb-141a-4376-b1aa-0725ef4e681a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:44.182973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8abe32eb-141a-4376-b1aa-0725ef4e681a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:44.265387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a9d19d155946b4bd0021ac54e62a3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.299848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a9d19d155946b4bd0021ac54e62a3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.301852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a9d19d155946b4bd0021ac54e62a3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.321143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a9d19d155946b4bd0021ac54e62a3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.322096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0cd471a5-5056-4269-83af-9baa2d8210f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a9d19d155946b4bd0021ac54e62a3f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:44.365981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 731/1451] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:43 2022] DELETE /identity/v3/users/b3a9d19d155946b4bd0021ac54e62a3f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:44.373115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58de1-adc9-4c16-85f8-d2ea40c3f89e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:44.374349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58de1-adc9-4c16-85f8-d2ea40c3f89e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:44.374656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b58de1-adc9-4c16-85f8-d2ea40c3f89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:44.375606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe32eb-141a-4376-b1aa-0725ef4e681a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:44.377415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abe32eb-141a-4376-b1aa-0725ef4e681a service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:44.407518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 721/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:44.419733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154969d-2224-4ded-80f4-b6bcae276842 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:44.423665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154969d-2224-4ded-80f4-b6bcae276842 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:44.424237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154969d-2224-4ded-80f4-b6bcae276842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:44.434689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-41b58de1-adc9-4c16-85f8-d2ea40c3f89e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:44.439348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-41b58de1-adc9-4c16-85f8-d2ea40c3f89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:44.493788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5154969d-2224-4ded-80f4-b6bcae276842 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:44.494892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5154969d-2224-4ded-80f4-b6bcae276842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:44.676640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 732/1453] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:44.676640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b58de1-adc9-4c16-85f8-d2ea40c3f89e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:44.679946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-41b58de1-adc9-4c16-85f8-d2ea40c3f89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:44.688670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 722/1454] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:44.690303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716cad4-caa3-4703-b302-a452a8acdb3b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:44.690608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716cad4-caa3-4703-b302-a452a8acdb3b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:44.690896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716cad4-caa3-4703-b302-a452a8acdb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:44.736720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4716cad4-caa3-4703-b302-a452a8acdb3b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:44.738342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4716cad4-caa3-4703-b302-a452a8acdb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:44.763792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 723/1455] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:44.765334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716cad4-caa3-4703-b302-a452a8acdb3b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:44.785541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716cad4-caa3-4703-b302-a452a8acdb3b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:44.800798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:44.801439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:44.801726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:44.802002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:44.802762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:44.803284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:44.805038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:44.838764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 724/1456] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:44.840854 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:44.842556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b58de1-adc9-4c16-85f8-d2ea40c3f89e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:44.850315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:44.854888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:44.855735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:44.856022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:44.856302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:44.862457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:45.005031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:45.040231 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:45.041969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:45.056696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4319e-a45d-42bd-8657-e3101b4309e2 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:45.068209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100207e4-b4e4-4de2-9011-ed2c55bec2dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:45.068846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100207e4-b4e4-4de2-9011-ed2c55bec2dc service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:45.069134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100207e4-b4e4-4de2-9011-ed2c55bec2dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:45.069422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100207e4-b4e4-4de2-9011-ed2c55bec2dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:45.082306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 733/1457] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:45.084524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:45.091216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100207e4-b4e4-4de2-9011-ed2c55bec2dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:45.100830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100207e4-b4e4-4de2-9011-ed2c55bec2dc service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:45.115215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4f466-e9bd-4cf2-8a19-25dfeaf719ab service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:45.132568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 725/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:45.137406 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa2dc-ef6d-4f19-8d00-2bdd9bf74f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:45.138247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa2dc-ef6d-4f19-8d00-2bdd9bf74f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:45.148546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa2dc-ef6d-4f19-8d00-2bdd9bf74f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:45.150197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa2dc-ef6d-4f19-8d00-2bdd9bf74f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:45.151146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfa2dc-ef6d-4f19-8d00-2bdd9bf74f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:45.154837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfa2dc-ef6d-4f19-8d00-2bdd9bf74f60 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:45.283463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 734/1459] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:45 2022] POST /identity/v3/projects => generated 369 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:45.285018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100207e4-b4e4-4de2-9011-ed2c55bec2dc service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:45.301082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100207e4-b4e4-4de2-9011-ed2c55bec2dc service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:45.319599 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:45.473989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cde26-3bff-4e08-a0a1-9645257e06cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:45.475231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cde26-3bff-4e08-a0a1-9645257e06cf admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:45.476422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cde26-3bff-4e08-a0a1-9645257e06cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:45.477631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cde26-3bff-4e08-a0a1-9645257e06cf admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:45.478864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cde26-3bff-4e08-a0a1-9645257e06cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:45.481610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5cde26-3bff-4e08-a0a1-9645257e06cf admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:45.531637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 726/1460] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:45 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:45.533457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa2dc-ef6d-4f19-8d00-2bdd9bf74f60 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:45.547216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa2dc-ef6d-4f19-8d00-2bdd9bf74f60 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:45.553970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4dcf78-eb23-4e86-89e7-57d74d027053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:45.554798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dcf78-eb23-4e86-89e7-57d74d027053 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:45.555296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dcf78-eb23-4e86-89e7-57d74d027053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:45.555730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4dcf78-eb23-4e86-89e7-57d74d027053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:45.556951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4dcf78-eb23-4e86-89e7-57d74d027053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:45.558773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4dcf78-eb23-4e86-89e7-57d74d027053 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:45.578402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 735/1461] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:45.579960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cde26-3bff-4e08-a0a1-9645257e06cf admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:45.595383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5cde26-3bff-4e08-a0a1-9645257e06cf admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:45.601012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fdcf7-b201-45cb-8d7c-611d44524b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:45.601890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fdcf7-b201-45cb-8d7c-611d44524b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:45.602285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fdcf7-b201-45cb-8d7c-611d44524b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:45.602676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fdcf7-b201-45cb-8d7c-611d44524b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:45.606047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fdcf7-b201-45cb-8d7c-611d44524b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:45.606047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fdcf7-b201-45cb-8d7c-611d44524b3b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:45.633895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 727/1462] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:45.635512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4dcf78-eb23-4e86-89e7-57d74d027053 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:45.653095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4dcf78-eb23-4e86-89e7-57d74d027053 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:45.667909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:45.668649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:45.668975 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:45.669272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] PATH_INFO: `/v3/projects/ce1f41e7b8244116ac18b3cebf7b51a7/users/edaa9636d5ba4972a70ba8cb15415ad8/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:45.712603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce1f41e7b8244116ac18b3cebf7b51a7, user_id=edaa9636d5ba4972a70ba8cb15415ad8, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:45.715473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:45.750341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 736/1463] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:45 2022] PUT /identity/v3/projects/ce1f41e7b8244116ac18b3cebf7b51a7/users/edaa9636d5ba4972a70ba8cb15415ad8/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:45.760994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b6ce5-4cca-41da-b3c5-cb77bd331f93 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:45.761270 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b6ce5-4cca-41da-b3c5-cb77bd331f93 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:45.761459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b6ce5-4cca-41da-b3c5-cb77bd331f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:45.830921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4a0b6ce5-4cca-41da-b3c5-cb77bd331f93 None None] MFA Rules not processed for user `edaa9636d5ba4972a70ba8cb15415ad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:45.834105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4a0b6ce5-4cca-41da-b3c5-cb77bd331f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:45.949214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:45.951280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:46.130000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 728/1464] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:46.142831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b6ce5-4cca-41da-b3c5-cb77bd331f93 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:46.179954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc3d3a-fc22-402f-844d-4e14462f733d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:46.196465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:46.197156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:46.197445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:46.197727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] PATH_INFO: `/v3/users/f443a8f611904179b1b3c425545265f4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:46.202173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b6ce5-4cca-41da-b3c5-cb77bd331f93 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:46.203381 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f443a8f611904179b1b3c425545265f4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:46.205377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:46.218000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2936c0-6c83-4c02-8eef-1520723dbe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:46.218643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2936c0-6c83-4c02-8eef-1520723dbe6d admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:46.221784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2936c0-6c83-4c02-8eef-1520723dbe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:46.222543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2936c0-6c83-4c02-8eef-1520723dbe6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:46.223276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2936c0-6c83-4c02-8eef-1520723dbe6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:46.236226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2936c0-6c83-4c02-8eef-1520723dbe6d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:46.353035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 729/1465] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:46 2022] POST /identity/v3/projects => generated 367 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:46.354312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2936c0-6c83-4c02-8eef-1520723dbe6d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:46.375860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f443a8f611904179b1b3c425545265f4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:46.410686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f443a8f611904179b1b3c425545265f4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:46.415555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f443a8f611904179b1b3c425545265f4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:46.427730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f443a8f611904179b1b3c425545265f4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:46.428731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c2239f86-0e94-40f9-8f77-5d21ff70fe0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f443a8f611904179b1b3c425545265f4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:46.468748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 737/1466] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:45 2022] DELETE /identity/v3/users/f443a8f611904179b1b3c425545265f4 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:12:46.619696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2936c0-6c83-4c02-8eef-1520723dbe6d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:46.632554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:46.694602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de591d9d-d95a-4777-a307-6bd46a8e650d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:46.695139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de591d9d-d95a-4777-a307-6bd46a8e650d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:46.695600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de591d9d-d95a-4777-a307-6bd46a8e650d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:46.724199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:46.756912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-de591d9d-d95a-4777-a307-6bd46a8e650d None None] MFA Rules not processed for user `9c414b04fe0544e3a938661bc957c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:46.763848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-de591d9d-d95a-4777-a307-6bd46a8e650d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:46.785094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:46.785732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:46.786064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:46.786273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:46.787044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:46.789610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:46.834089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 730/1467] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:46 2022] POST /identity/v3/users => generated 290 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:46.834089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:46.841431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:46.882321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b350f11-ee71-468a-beff-2d49b9760777 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:46.890733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c6bfa-bde0-4815-a1f4-c7fe39b8ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:46.891573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6bfa-bde0-4815-a1f4-c7fe39b8ac27 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:46.892054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6bfa-bde0-4815-a1f4-c7fe39b8ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:46.892662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6bfa-bde0-4815-a1f4-c7fe39b8ac27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:46.899990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c6bfa-bde0-4815-a1f4-c7fe39b8ac27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:46.902071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c6bfa-bde0-4815-a1f4-c7fe39b8ac27 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:46.926673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 731/1468] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:46.933152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c6bfa-bde0-4815-a1f4-c7fe39b8ac27 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:46.958344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c6bfa-bde0-4815-a1f4-c7fe39b8ac27 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:46.965780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:46.967220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:46.967776 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:46.968667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] PATH_INFO: `/v3/projects/0a3889a79a674ec98c66bbc7e3cec104/users/95b6b923e026462eae1d7ac1fec4612c/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:47.004066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3889a79a674ec98c66bbc7e3cec104, user_id=95b6b923e026462eae1d7ac1fec4612c, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:47.015896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 738/1469] 10.222.0.25 () {58 vars in 1015 bytes} [Mon Oct 24 13:12:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:12:47.020135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:47.059047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 732/1470] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:46 2022] PUT /identity/v3/projects/0a3889a79a674ec98c66bbc7e3cec104/users/95b6b923e026462eae1d7ac1fec4612c/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:47.060346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de591d9d-d95a-4777-a307-6bd46a8e650d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:47.086253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de591d9d-d95a-4777-a307-6bd46a8e650d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:47.160577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:47.162473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:47.275921 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb8eb-ff28-4580-951d-2afab9b551f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:47.277806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb8eb-ff28-4580-951d-2afab9b551f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:47.278297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb8eb-ff28-4580-951d-2afab9b551f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:47.278758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28efb8eb-ff28-4580-951d-2afab9b551f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:47.279654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb8eb-ff28-4580-951d-2afab9b551f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:47.281877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28efb8eb-ff28-4580-951d-2afab9b551f6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:47.300081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 739/1471] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:47.304900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb8eb-ff28-4580-951d-2afab9b551f6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:47.346460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28efb8eb-ff28-4580-951d-2afab9b551f6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:47.362791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c5a5a-7baa-422b-85a4-870d2780bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:47.363957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c5a5a-7baa-422b-85a4-870d2780bff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:47.366771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c5a5a-7baa-422b-85a4-870d2780bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:47.368232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c5a5a-7baa-422b-85a4-870d2780bff9 admin admin] PATH_INFO: `/v3/projects/0a3889a79a674ec98c66bbc7e3cec104/users/95b6b923e026462eae1d7ac1fec4612c/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:47.374813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a9c72-b82b-4f92-b179-e0dba9a31c22 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:47.375852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c5a5a-7baa-422b-85a4-870d2780bff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a3889a79a674ec98c66bbc7e3cec104, user_id=95b6b923e026462eae1d7ac1fec4612c, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:47.393058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:47.393949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:47.394380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:47.394836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:47.397167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c5a5a-7baa-422b-85a4-870d2780bff9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:47.426202 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:47.428222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:47.455022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 740/1472] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:47 2022] PUT /identity/v3/projects/0a3889a79a674ec98c66bbc7e3cec104/users/95b6b923e026462eae1d7ac1fec4612c/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:47.455837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 733/1473] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:12:47.470160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a130691-9d89-498b-b3ef-ca27711f7301 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:47.473894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a130691-9d89-498b-b3ef-ca27711f7301 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:47.474455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a130691-9d89-498b-b3ef-ca27711f7301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:47.585469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0a130691-9d89-498b-b3ef-ca27711f7301 None None] MFA Rules not processed for user `95b6b923e026462eae1d7ac1fec4612c`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:47.588329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0a130691-9d89-498b-b3ef-ca27711f7301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:47.787036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 741/1474] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:47.802122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:47.803746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:47.983164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a169fb-fdb9-4d34-928a-0cb5bce46029 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:48.046225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:48.160361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02605c7b-09d3-471f-97a2-81cd19f0a8c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:48.161224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02605c7b-09d3-471f-97a2-81cd19f0a8c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:48.161811 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02605c7b-09d3-471f-97a2-81cd19f0a8c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:48.162212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02605c7b-09d3-471f-97a2-81cd19f0a8c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:48.183393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02605c7b-09d3-471f-97a2-81cd19f0a8c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:48.185231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02605c7b-09d3-471f-97a2-81cd19f0a8c3 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:48.198606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 734/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:12:48.343413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a130691-9d89-498b-b3ef-ca27711f7301 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:48.344868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0a130691-9d89-498b-b3ef-ca27711f7301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:48.363343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a130691-9d89-498b-b3ef-ca27711f7301 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:48.390917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:48.391593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:48.391876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:48.392155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:48.394586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:48.459361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:48.560847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:48.564083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:48.579466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 742/1476] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:48.751445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097e632-1fb8-42a4-8040-eb7a2575451e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:48.752114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097e632-1fb8-42a4-8040-eb7a2575451e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:48.752913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097e632-1fb8-42a4-8040-eb7a2575451e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:48.807728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6097e632-1fb8-42a4-8040-eb7a2575451e None None] MFA Rules not processed for user `c5619c8183e04e1d9073f1b40e057967`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:48.813017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6097e632-1fb8-42a4-8040-eb7a2575451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:49.067423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 735/1477] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:49.291611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:49.293054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:49.478646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d5be5-b1cb-4ec5-97ac-1ef5a2768fc5 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:49.494321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:49.497952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:49.500058 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:49.500563 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] PATH_INFO: `/v3/users/0d7515577a5b49d99d347c3c8c401b7a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:49.506008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7515577a5b49d99d347c3c8c401b7a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:49.514130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:49.584162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e54e-554b-42bd-9a58-97f8c5317d4e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:49.584713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e54e-554b-42bd-9a58-97f8c5317d4e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:49.584713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e54e-554b-42bd-9a58-97f8c5317d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:49.644445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9770e54e-554b-42bd-9a58-97f8c5317d4e None None] MFA Rules not processed for user `3e02f6f701324af6b53762569f1a9d21`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:49.650282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9770e54e-554b-42bd-9a58-97f8c5317d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:49.708168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7515577a5b49d99d347c3c8c401b7a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:49.774786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7515577a5b49d99d347c3c8c401b7a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:49.775483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7515577a5b49d99d347c3c8c401b7a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:49.795351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7515577a5b49d99d347c3c8c401b7a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:49.796784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7515577a5b49d99d347c3c8c401b7a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:49.835375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 736/1478] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:49.868483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 743/1479] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:49 2022] DELETE /identity/v3/users/0d7515577a5b49d99d347c3c8c401b7a => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:49.872767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cec1ba-33c2-45f5-b476-50aa2c097ed2 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:49.873062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cec1ba-33c2-45f5-b476-50aa2c097ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:49.873375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cec1ba-33c2-45f5-b476-50aa2c097ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:49.925211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-86cec1ba-33c2-45f5-b476-50aa2c097ed2 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:49.931603 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-86cec1ba-33c2-45f5-b476-50aa2c097ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:50.108134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 737/1480] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:50.160291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:50.163318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:50.288729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:50.305838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e356d-0dad-49f1-8c5c-2f638e9f8ab0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:50.311821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65fbfd-7107-4157-9bff-d0cacf35008f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:50.313961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65fbfd-7107-4157-9bff-d0cacf35008f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:50.315247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65fbfd-7107-4157-9bff-d0cacf35008f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:50.315608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f65fbfd-7107-4157-9bff-d0cacf35008f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:50.345691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65fbfd-7107-4157-9bff-d0cacf35008f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:50.350114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f65fbfd-7107-4157-9bff-d0cacf35008f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:50.380711 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 744/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:50.381580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cec1ba-33c2-45f5-b476-50aa2c097ed2 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:50.383332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-86cec1ba-33c2-45f5-b476-50aa2c097ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:50.415165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cec1ba-33c2-45f5-b476-50aa2c097ed2 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:50.429375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:50.432014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:50.432325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:50.432728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] PATH_INFO: `/v3/projects/c55ee1329da34912b0cb4df30bc11d2c` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:50.436044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c55ee1329da34912b0cb4df30bc11d2c)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:50.441009 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:50.479927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:50.560630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c55ee1329da34912b0cb4df30bc11d2c', 'request_id': 'req-f2f7c88c-179c-462f-8326-aced6e929778'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:50.576923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c55ee1329da34912b0cb4df30bc11d2c', 'request_id': 'req-f2f7c88c-179c-462f-8326-aced6e929778'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:50.587215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c55ee1329da34912b0cb4df30bc11d2c', 'request_id': 'req-f2f7c88c-179c-462f-8326-aced6e929778'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:50.607496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] The token cache is being invalidate because project c55ee1329da34912b0cb4df30bc11d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:50.608968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:50.610273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:50.621317 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 738/1482] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:50 2022] DELETE /identity/v3/projects/c55ee1329da34912b0cb4df30bc11d2c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:50.621815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65fbfd-7107-4157-9bff-d0cacf35008f service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:50.638700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f65fbfd-7107-4157-9bff-d0cacf35008f service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:50.714543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:50.718115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:50.850076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:50.850740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:50.851379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:50.851672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] PATH_INFO: `/v3/users/aaf859d57bfe47bdb28d8c44b2d52137` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:50.853275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf859d57bfe47bdb28d8c44b2d52137)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:50.868881 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:50.937267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7c88c-179c-462f-8326-aced6e929778 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:50.956465 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:50.959100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:50.972950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:50.974277 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] PATH_INFO: `/v3/projects/07e225871dfc4e7587af04d7ca61f24b` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:50.980087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07e225871dfc4e7587af04d7ca61f24b)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:50.986165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:51.045470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf859d57bfe47bdb28d8c44b2d52137'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.129002 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf859d57bfe47bdb28d8c44b2d52137'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.131302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf859d57bfe47bdb28d8c44b2d52137'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.159566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf859d57bfe47bdb28d8c44b2d52137'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.161472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf859d57bfe47bdb28d8c44b2d52137'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.169022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07e225871dfc4e7587af04d7ca61f24b', 'request_id': 'req-689cbbc2-310d-4f04-b986-27cc71e05b30'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.197649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07e225871dfc4e7587af04d7ca61f24b', 'request_id': 'req-689cbbc2-310d-4f04-b986-27cc71e05b30'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.205673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 745/1483] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:50 2022] DELETE /identity/v3/users/aaf859d57bfe47bdb28d8c44b2d52137 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:51.220602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07e225871dfc4e7587af04d7ca61f24b', 'request_id': 'req-689cbbc2-310d-4f04-b986-27cc71e05b30'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.242977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] The token cache is being invalidate because project 07e225871dfc4e7587af04d7ca61f24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:51.243343 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.244080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-689cbbc2-310d-4f04-b986-27cc71e05b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.251948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 739/1484] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:50 2022] DELETE /identity/v3/projects/07e225871dfc4e7587af04d7ca61f24b => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:51.253298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:51.255090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:51.444109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff15eb5-92c2-4cbf-b245-fdd3f7bab3ea admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:51.459898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:51.460576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:51.460866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:51.461148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] PATH_INFO: `/v3/users/01f050da05ae47ae8731bb1cd4abfdd7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:51.462510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f050da05ae47ae8731bb1cd4abfdd7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:51.468413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:51.482054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:51.641628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f050da05ae47ae8731bb1cd4abfdd7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.684878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f050da05ae47ae8731bb1cd4abfdd7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.687353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f050da05ae47ae8731bb1cd4abfdd7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.716249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f050da05ae47ae8731bb1cd4abfdd7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.717233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d0b06aec-3537-45dc-b486-ff88aa5463dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f050da05ae47ae8731bb1cd4abfdd7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:51.762143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 746/1485] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:51 2022] DELETE /identity/v3/users/01f050da05ae47ae8731bb1cd4abfdd7 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:52.204778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1add475-a32e-4c45-8e32-47cc34f9fbd5 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:52.205295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1add475-a32e-4c45-8e32-47cc34f9fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:52.205369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1add475-a32e-4c45-8e32-47cc34f9fbd5 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:52.207551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 740/1486] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:12:52.493076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac37214-a77e-425d-ae86-e3113e1db1e3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:52.493618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac37214-a77e-425d-ae86-e3113e1db1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:52.493850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac37214-a77e-425d-ae86-e3113e1db1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:52.552553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9ac37214-a77e-425d-ae86-e3113e1db1e3 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:52.553979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9ac37214-a77e-425d-ae86-e3113e1db1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:52.820545 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 747/1487] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:12:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:53.302599 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1add475-a32e-4c45-8e32-47cc34f9fbd5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:53.305448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d1add475-a32e-4c45-8e32-47cc34f9fbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:53.412683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:53.497083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1add475-a32e-4c45-8e32-47cc34f9fbd5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:53.523601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:53.524352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:53.525128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:53.525128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] PATH_INFO: `/v3/projects/a96b5da347f744c9b305f206acb396fe` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:53.534344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96b5da347f744c9b305f206acb396fe)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:53.541122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:53.694706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac37214-a77e-425d-ae86-e3113e1db1e3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:53.696396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9ac37214-a77e-425d-ae86-e3113e1db1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:53.711291 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96b5da347f744c9b305f206acb396fe', 'request_id': 'req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:53.735133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac37214-a77e-425d-ae86-e3113e1db1e3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:53.743405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96b5da347f744c9b305f206acb396fe', 'request_id': 'req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:53.775098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96b5da347f744c9b305f206acb396fe', 'request_id': 'req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:53.824058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] The token cache is being invalidate because project a96b5da347f744c9b305f206acb396fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:53.824365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:53.826060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:53.828285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 741/1488] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:53 2022] DELETE /identity/v3/projects/a96b5da347f744c9b305f206acb396fe => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:53.829087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:53.833567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:53.940911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:53.941985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:53.942278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:53.942558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] PATH_INFO: `/v3/projects/391ed02c7029442f8acedd097b32bc76` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:53.944052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=391ed02c7029442f8acedd097b32bc76)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:53.946050 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:54.010134 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:54.118278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391ed02c7029442f8acedd097b32bc76', 'request_id': 'req-219957da-7455-4f27-a887-96f66f80c7cf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.132102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391ed02c7029442f8acedd097b32bc76', 'request_id': 'req-219957da-7455-4f27-a887-96f66f80c7cf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.143251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391ed02c7029442f8acedd097b32bc76', 'request_id': 'req-219957da-7455-4f27-a887-96f66f80c7cf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.148472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae2563-3fb6-4cb7-abaa-0b84f126dd22 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:54.162098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] The token cache is being invalidate because project 391ed02c7029442f8acedd097b32bc76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:54.162409 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.163201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-219957da-7455-4f27-a887-96f66f80c7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.169641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 748/1489] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:53 2022] DELETE /identity/v3/projects/391ed02c7029442f8acedd097b32bc76 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:12:54.171700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3909f-4b6e-4186-afd7-aee5e3a8bc60 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.172202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3909f-4b6e-4186-afd7-aee5e3a8bc60 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.172540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3909f-4b6e-4186-afd7-aee5e3a8bc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.202953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b9e3909f-4b6e-4186-afd7-aee5e3a8bc60 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:54.204515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b9e3909f-4b6e-4186-afd7-aee5e3a8bc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.271543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:54.302274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:54.303545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.304119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.304119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.310449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 749/1490] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:54.310600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.310677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3909f-4b6e-4186-afd7-aee5e3a8bc60 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:54.310677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b9e3909f-4b6e-4186-afd7-aee5e3a8bc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.329166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:54.330857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:54.332324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3909f-4b6e-4186-afd7-aee5e3a8bc60 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:54.344019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 742/1491] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:54.346017 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:54.346673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.346956 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.347287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] PATH_INFO: `/v3/users/edaa9636d5ba4972a70ba8cb15415ad8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.350614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edaa9636d5ba4972a70ba8cb15415ad8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:54.352837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:54.369293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:54.380485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.424597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69357544-de3c-4bd2-b2a5-f602c0c5234f service nova] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:54.437830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:54.438883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.439259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.440185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] PATH_INFO: `/v3/projects/fecdd1980bcf4f9ea66d7e0724b868d0` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.450943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fecdd1980bcf4f9ea66d7e0724b868d0)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:54.456699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:54.475658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edaa9636d5ba4972a70ba8cb15415ad8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.515503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edaa9636d5ba4972a70ba8cb15415ad8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.516306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edaa9636d5ba4972a70ba8cb15415ad8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.534038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edaa9636d5ba4972a70ba8cb15415ad8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.536111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-796ec389-1c2b-4ba7-ac0e-45e1c22e60a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edaa9636d5ba4972a70ba8cb15415ad8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.555315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 750/1492] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:54 2022] DELETE /identity/v3/users/edaa9636d5ba4972a70ba8cb15415ad8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:54.556937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531621-4f1f-438d-b12c-e155b1fc98ca None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.573229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fecdd1980bcf4f9ea66d7e0724b868d0', 'request_id': 'req-2a41a04b-fd29-43d8-a024-8ee3ada34640'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.579804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531621-4f1f-438d-b12c-e155b1fc98ca None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.581957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e531621-4f1f-438d-b12c-e155b1fc98ca None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.586925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 751/1493] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:12:54 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:12:54.589445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fecdd1980bcf4f9ea66d7e0724b868d0', 'request_id': 'req-2a41a04b-fd29-43d8-a024-8ee3ada34640'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.590912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043cc8d-85f5-4b82-989b-1c23306fe3fb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.591719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043cc8d-85f5-4b82-989b-1c23306fe3fb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.593037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043cc8d-85f5-4b82-989b-1c23306fe3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.607847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fecdd1980bcf4f9ea66d7e0724b868d0', 'request_id': 'req-2a41a04b-fd29-43d8-a024-8ee3ada34640'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.659137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] The token cache is being invalidate because project fecdd1980bcf4f9ea66d7e0724b868d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:54.659901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.661317 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2a41a04b-fd29-43d8-a024-8ee3ada34640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:54.666281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 743/1494] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:54 2022] DELETE /identity/v3/projects/fecdd1980bcf4f9ea66d7e0724b868d0 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:54.666281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae89668-5e95-456c-862a-7e34b39b4828 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.666281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae89668-5e95-456c-862a-7e34b39b4828 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.667535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae89668-5e95-456c-862a-7e34b39b4828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.676037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5043cc8d-85f5-4b82-989b-1c23306fe3fb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:54.677120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5043cc8d-85f5-4b82-989b-1c23306fe3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.692299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:54.695018 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-dae89668-5e95-456c-862a-7e34b39b4828 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:54.696885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-dae89668-5e95-456c-862a-7e34b39b4828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.824203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 752/1495] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:54.827241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e462-671c-402c-9111-a8ea6e0c6b74 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.827623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e462-671c-402c-9111-a8ea6e0c6b74 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.828314 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495e462-671c-402c-9111-a8ea6e0c6b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.886985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 744/1496] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:54.889030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e423a-f10e-42e1-8ce7-7397a35d0e7a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.889357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e423a-f10e-42e1-8ce7-7397a35d0e7a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.889641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e423a-f10e-42e1-8ce7-7397a35d0e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.904086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c495e462-671c-402c-9111-a8ea6e0c6b74 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:54.905980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c495e462-671c-402c-9111-a8ea6e0c6b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.930488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 753/1497] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:54.930488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495e462-671c-402c-9111-a8ea6e0c6b74 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:54.933260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c495e462-671c-402c-9111-a8ea6e0c6b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.941705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a75e423a-f10e-42e1-8ce7-7397a35d0e7a None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:54.945773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a75e423a-f10e-42e1-8ce7-7397a35d0e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:54.961732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495e462-671c-402c-9111-a8ea6e0c6b74 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:54.975661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:54.976331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:54.976613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:54.976890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] PATH_INFO: `/v3/users/51b87de9e37e44f59ba9888d70ceb8eb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:54.980179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b87de9e37e44f59ba9888d70ceb8eb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:54.982143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.120581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:55.142934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 745/1498] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:12:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:55.143818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4dad6-9916-4b1b-acec-701e24d66b57 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.144253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4dad6-9916-4b1b-acec-701e24d66b57 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.144873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d4dad6-9916-4b1b-acec-701e24d66b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.146386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b87de9e37e44f59ba9888d70ceb8eb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:55.169485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b87de9e37e44f59ba9888d70ceb8eb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:55.171334 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b87de9e37e44f59ba9888d70ceb8eb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:55.180426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b87de9e37e44f59ba9888d70ceb8eb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:55.181396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b87de9e37e44f59ba9888d70ceb8eb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:55.196735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 754/1499] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:54 2022] DELETE /identity/v3/users/51b87de9e37e44f59ba9888d70ceb8eb => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:55.197586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.197998 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f2d4dad6-9916-4b1b-acec-701e24d66b57 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:55.199520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f2d4dad6-9916-4b1b-acec-701e24d66b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:55.204245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:55.310086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc22b25f-c619-4669-9fff-fb736f7d199b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.313554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 746/1500] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:55.316422 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.317376 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.317662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.317785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4dad6-9916-4b1b-acec-701e24d66b57 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.317948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.318821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:55.319408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f2d4dad6-9916-4b1b-acec-701e24d66b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:55.319832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.322457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.338073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 755/1501] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:55.338958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.341834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:55.368684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:55.414991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d4dad6-9916-4b1b-acec-701e24d66b57 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.422442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.424264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.424404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.424899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.426866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:55.443024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.444508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.445427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bf412-dbb4-44e2-b256-643d0f9f01bd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.454116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.454896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.455237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.455786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.457119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 747/1502] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:55.458229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.458823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:55.469062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422991b-1906-404c-a398-96f565f2b5a1 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.476690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.477607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.478186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.478644 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.478768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.479663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:55.480187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.480485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.492394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.493864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 756/1503] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:55.494601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.518043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b416a60-6fde-4807-a5cc-2014eb0e4f70 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.529792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 748/1504] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:55.530210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373052d-d892-4018-b582-581683db5261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.530210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373052d-d892-4018-b582-581683db5261 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.530210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373052d-d892-4018-b582-581683db5261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.530577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373052d-d892-4018-b582-581683db5261 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.532251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.532598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373052d-d892-4018-b582-581683db5261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.548520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373052d-d892-4018-b582-581683db5261 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.557477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dd642-8c65-4b9a-918e-fbb51c7482f9 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.573811 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bad3fb-1f6e-4822-b66a-c29e0c0986d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.575124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad3fb-1f6e-4822-b66a-c29e0c0986d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.575124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad3fb-1f6e-4822-b66a-c29e0c0986d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.575124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bad3fb-1f6e-4822-b66a-c29e0c0986d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.575756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bad3fb-1f6e-4822-b66a-c29e0c0986d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.579146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bad3fb-1f6e-4822-b66a-c29e0c0986d9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.609810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 757/1505] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:55 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:55.615891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373052d-d892-4018-b582-581683db5261 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.643146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 749/1506] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:55 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:12:55.647333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bad3fb-1f6e-4822-b66a-c29e0c0986d9 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.780292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373052d-d892-4018-b582-581683db5261 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.808706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bad3fb-1f6e-4822-b66a-c29e0c0986d9 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.814885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6edfa8-2d17-44bd-90ba-182c1670cc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.815687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6edfa8-2d17-44bd-90ba-182c1670cc25 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.816047 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6edfa8-2d17-44bd-90ba-182c1670cc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.816451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6edfa8-2d17-44bd-90ba-182c1670cc25 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.817210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6edfa8-2d17-44bd-90ba-182c1670cc25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.819635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6edfa8-2d17-44bd-90ba-182c1670cc25 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.830413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:55.854261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 750/1507] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:55 2022] POST /identity/v3/users => generated 288 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:12:55.855910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6edfa8-2d17-44bd-90ba-182c1670cc25 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.876833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6edfa8-2d17-44bd-90ba-182c1670cc25 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.892996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ed52d-06fa-4096-b6c4-78b42d6a1fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.893870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ed52d-06fa-4096-b6c4-78b42d6a1fff admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.894191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ed52d-06fa-4096-b6c4-78b42d6a1fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.894554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ed52d-06fa-4096-b6c4-78b42d6a1fff admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.895253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ed52d-06fa-4096-b6c4-78b42d6a1fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.897022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ed52d-06fa-4096-b6c4-78b42d6a1fff admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.912590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 751/1508] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:55.916281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ed52d-06fa-4096-b6c4-78b42d6a1fff admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.933721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ed52d-06fa-4096-b6c4-78b42d6a1fff admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.941443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.942320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.942493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.942762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.943939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.945261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.950798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef5abd-d873-4921-8a0e-d31b0483a711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.958611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5abd-d873-4921-8a0e-d31b0483a711 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.958899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5abd-d873-4921-8a0e-d31b0483a711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.959212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5abd-d873-4921-8a0e-d31b0483a711 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.959689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 752/1509] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:55.960617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:55.962341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:55.963543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef5abd-d873-4921-8a0e-d31b0483a711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.965431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef5abd-d873-4921-8a0e-d31b0483a711 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.977317 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104c18b-e8d0-4c1a-9f49-3c9b5e7e8901 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:55.982666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:55.985147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:55.985462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:55.985766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] PATH_INFO: `/v3/projects/3060d830aa424d0284f2c59eb096ae55` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:55.988918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3060d830aa424d0284f2c59eb096ae55)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:55.991392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:55.997005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 758/1510] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:55 2022] POST /identity/v3/users => generated 287 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:56.001397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef5abd-d873-4921-8a0e-d31b0483a711 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.012050 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef5abd-d873-4921-8a0e-d31b0483a711 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.017171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609135b-deec-4e56-af46-813c3f6fd9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:56.017885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609135b-deec-4e56-af46-813c3f6fd9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.018992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609135b-deec-4e56-af46-813c3f6fd9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.018992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609135b-deec-4e56-af46-813c3f6fd9c8 admin admin] PATH_INFO: `/v3/projects/bf3c13ae2890450ea18850ac2eecde74/users/5616c0f1b99c4a6bbe0adf0ea0e130f8/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.039318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609135b-deec-4e56-af46-813c3f6fd9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3c13ae2890450ea18850ac2eecde74, user_id=5616c0f1b99c4a6bbe0adf0ea0e130f8, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:56.042011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609135b-deec-4e56-af46-813c3f6fd9c8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:56.044630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:56.056795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3060d830aa424d0284f2c59eb096ae55', 'request_id': 'req-26e36af9-258a-4d2a-87a4-fc0d7373f438'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.062235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 759/1511] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:56 2022] PUT /identity/v3/projects/bf3c13ae2890450ea18850ac2eecde74/users/5616c0f1b99c4a6bbe0adf0ea0e130f8/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:56.064909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609135b-deec-4e56-af46-813c3f6fd9c8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.075746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3060d830aa424d0284f2c59eb096ae55', 'request_id': 'req-26e36af9-258a-4d2a-87a4-fc0d7373f438'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.090856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3060d830aa424d0284f2c59eb096ae55', 'request_id': 'req-26e36af9-258a-4d2a-87a4-fc0d7373f438'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.110928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] The token cache is being invalidate because project 3060d830aa424d0284f2c59eb096ae55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:56.111388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.113565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26e36af9-258a-4d2a-87a4-fc0d7373f438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.125284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 753/1512] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:55 2022] DELETE /identity/v3/projects/3060d830aa424d0284f2c59eb096ae55 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:56.125591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937370-f4e2-4d0c-bdbb-6f368e202b59 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.125880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937370-f4e2-4d0c-bdbb-6f368e202b59 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.126155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937370-f4e2-4d0c-bdbb-6f368e202b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.159126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609135b-deec-4e56-af46-813c3f6fd9c8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.165105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:56.166088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.166584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.167104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.168170 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:56.171446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:56.182893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 760/1513] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:56.184187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.186134 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:56.189425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e4937370-f4e2-4d0c-bdbb-6f368e202b59 None None] MFA Rules not processed for user `5616c0f1b99c4a6bbe0adf0ea0e130f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:56.190610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e4937370-f4e2-4d0c-bdbb-6f368e202b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:56.202712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb52539-b4cc-4056-a3c7-f50b14aca270 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.207448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:56.208082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.208369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.208642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] PATH_INFO: `/v3/projects/ce1f41e7b8244116ac18b3cebf7b51a7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.209777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce1f41e7b8244116ac18b3cebf7b51a7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:56.211775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:56.274935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce1f41e7b8244116ac18b3cebf7b51a7', 'request_id': 'req-abc9a494-0f9d-4b53-9d40-a99625d4aa74'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.284726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce1f41e7b8244116ac18b3cebf7b51a7', 'request_id': 'req-abc9a494-0f9d-4b53-9d40-a99625d4aa74'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.287788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 754/1514] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:56.289678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1885e-1541-451d-bb72-982d21a2cee8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.290054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1885e-1541-451d-bb72-982d21a2cee8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.290432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1885e-1541-451d-bb72-982d21a2cee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.297570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce1f41e7b8244116ac18b3cebf7b51a7', 'request_id': 'req-abc9a494-0f9d-4b53-9d40-a99625d4aa74'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.309868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] The token cache is being invalidate because project ce1f41e7b8244116ac18b3cebf7b51a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:56.310163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.310853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.316302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 761/1515] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:56 2022] DELETE /identity/v3/projects/ce1f41e7b8244116ac18b3cebf7b51a7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:56.316583 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0ce1885e-1541-451d-bb72-982d21a2cee8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:56.317034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.317155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0ce1885e-1541-451d-bb72-982d21a2cee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:56.320244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:56.369280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:56.393166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 755/1516] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:56.393856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9a494-0f9d-4b53-9d40-a99625d4aa74 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.394003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1885e-1541-451d-bb72-982d21a2cee8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.398456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:56.399068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.399458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.399731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.400345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0ce1885e-1541-451d-bb72-982d21a2cee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:56.400440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:56.401998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:56.415963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 762/1517] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:56.415963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.415963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:56.428095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36c6e24-f98b-4094-ba76-b7abe36c74a6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.433223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:56.433858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.434143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.434420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] PATH_INFO: `/v3/users/95b6b923e026462eae1d7ac1fec4612c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.435798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95b6b923e026462eae1d7ac1fec4612c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:56.438461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:56.441778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:56.541654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1885e-1541-451d-bb72-982d21a2cee8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.546332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95b6b923e026462eae1d7ac1fec4612c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.554348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:56.555836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.556453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.556921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.560713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:56.577852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95b6b923e026462eae1d7ac1fec4612c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.578440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95b6b923e026462eae1d7ac1fec4612c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.588610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95b6b923e026462eae1d7ac1fec4612c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.589443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ba584c44-4a4c-4b2e-acfc-3df5c43ad005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95b6b923e026462eae1d7ac1fec4612c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:56.611288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 763/1518] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:12:56 2022] DELETE /identity/v3/users/95b6b923e026462eae1d7ac1fec4612c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:56.614650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547170c-bb03-4164-8bb5-b9861bdba939 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.614650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547170c-bb03-4164-8bb5-b9861bdba939 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.614650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547170c-bb03-4164-8bb5-b9861bdba939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.641039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6547170c-bb03-4164-8bb5-b9861bdba939 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:56.642353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6547170c-bb03-4164-8bb5-b9861bdba939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:56.654576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:56.660716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:56.681647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 756/1519] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:56.682333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.704939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d814f-e22c-4273-a3d4-97de4c94802e service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.738054 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 764/1520] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:56.739573 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547170c-bb03-4164-8bb5-b9861bdba939 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.753084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547170c-bb03-4164-8bb5-b9861bdba939 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.764495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:56.765700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.765996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.766275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.766961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:56.767343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:56.770098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:56.793775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 765/1521] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:12:56.794908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:56.809836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69162a0-bc6d-4875-b628-c887b75ebee1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:56.817524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc7b1a-78d1-4dbf-855a-e7bcde053a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:56.818548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc7b1a-78d1-4dbf-855a-e7bcde053a38 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:56.818996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc7b1a-78d1-4dbf-855a-e7bcde053a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:56.819774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc7b1a-78d1-4dbf-855a-e7bcde053a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:56.820535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc7b1a-78d1-4dbf-855a-e7bcde053a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:56.823277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dc7b1a-78d1-4dbf-855a-e7bcde053a38 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:56.894743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 766/1522] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:56 2022] POST /identity/v3/projects => generated 399 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:56.895974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc7b1a-78d1-4dbf-855a-e7bcde053a38 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.012131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:57.040518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55a138-1d9d-4f5a-b44d-cd554f131975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.046971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55a138-1d9d-4f5a-b44d-cd554f131975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.048168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55a138-1d9d-4f5a-b44d-cd554f131975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.049173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55a138-1d9d-4f5a-b44d-cd554f131975 admin admin] PATH_INFO: `/v3/projects/ce35ae5204c74b549f2bb6b8f7078e9b/users/0cf7d374437a4de8b70ee43bba2ab5bb/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.051530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc7b1a-78d1-4dbf-855a-e7bcde053a38 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.068646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482bd2c-9489-4770-bf14-3313047d4985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.069274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482bd2c-9489-4770-bf14-3313047d4985 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.069639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482bd2c-9489-4770-bf14-3313047d4985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.070078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482bd2c-9489-4770-bf14-3313047d4985 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.070883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482bd2c-9489-4770-bf14-3313047d4985 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.080676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482bd2c-9489-4770-bf14-3313047d4985 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.095251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55a138-1d9d-4f5a-b44d-cd554f131975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce35ae5204c74b549f2bb6b8f7078e9b, user_id=0cf7d374437a4de8b70ee43bba2ab5bb, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.098058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55a138-1d9d-4f5a-b44d-cd554f131975 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.123979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 757/1523] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:56 2022] PUT /identity/v3/projects/ce35ae5204c74b549f2bb6b8f7078e9b/users/0cf7d374437a4de8b70ee43bba2ab5bb/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:57.135866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431363cf-4d5c-4f6e-9f6b-9c7636989e32 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.136280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431363cf-4d5c-4f6e-9f6b-9c7636989e32 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.136668 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431363cf-4d5c-4f6e-9f6b-9c7636989e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.138800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 767/1524] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:56 2022] POST /identity/v3/users => generated 300 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:12:57.141347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482bd2c-9489-4770-bf14-3313047d4985 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.165362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482bd2c-9489-4770-bf14-3313047d4985 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.204407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-431363cf-4d5c-4f6e-9f6b-9c7636989e32 None None] MFA Rules not processed for user `0cf7d374437a4de8b70ee43bba2ab5bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:57.205885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-431363cf-4d5c-4f6e-9f6b-9c7636989e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:57.221798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:57.312736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1b97-2969-415f-9b7a-ed0a419fa56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.314911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1b97-2969-415f-9b7a-ed0a419fa56f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.316604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1b97-2969-415f-9b7a-ed0a419fa56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.317073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d1b97-2969-415f-9b7a-ed0a419fa56f admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.318032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d1b97-2969-415f-9b7a-ed0a419fa56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.321524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d1b97-2969-415f-9b7a-ed0a419fa56f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.353211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 768/1525] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:57.357806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1b97-2969-415f-9b7a-ed0a419fa56f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.377603 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 758/1526] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:12:57.382005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431363cf-4d5c-4f6e-9f6b-9c7636989e32 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.387690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-431363cf-4d5c-4f6e-9f6b-9c7636989e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:57.418369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d1b97-2969-415f-9b7a-ed0a419fa56f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.419660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431363cf-4d5c-4f6e-9f6b-9c7636989e32 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.429898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc7b8f-49de-4bba-9604-8df301ff11ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.430171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be7b56-6363-4da8-994f-32d88bfbf9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.430691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc7b8f-49de-4bba-9604-8df301ff11ce admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.431043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be7b56-6363-4da8-994f-32d88bfbf9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.431586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc7b8f-49de-4bba-9604-8df301ff11ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.431870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc7b8f-49de-4bba-9604-8df301ff11ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.432300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be7b56-6363-4da8-994f-32d88bfbf9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.432596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc7b8f-49de-4bba-9604-8df301ff11ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.433516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be7b56-6363-4da8-994f-32d88bfbf9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.435513 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be7b56-6363-4da8-994f-32d88bfbf9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.439694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc7b8f-49de-4bba-9604-8df301ff11ce admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.440668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be7b56-6363-4da8-994f-32d88bfbf9f0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.468900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 769/1527] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:57.474806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be7b56-6363-4da8-994f-32d88bfbf9f0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.505428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be7b56-6363-4da8-994f-32d88bfbf9f0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.513644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 759/1528] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:57 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:12:57.516710 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe661f64-992f-45c6-8526-2c242576947c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.517859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc7b8f-49de-4bba-9604-8df301ff11ce admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.518859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe661f64-992f-45c6-8526-2c242576947c admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.519669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe661f64-992f-45c6-8526-2c242576947c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.521434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe661f64-992f-45c6-8526-2c242576947c admin admin] PATH_INFO: `/v3/projects/13bdc95ac164408b9114a5cb69c9467a/users/729295e0893a4c989e4a1113691f7491/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.559812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe661f64-992f-45c6-8526-2c242576947c admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bdc95ac164408b9114a5cb69c9467a, user_id=729295e0893a4c989e4a1113691f7491, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.562699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe661f64-992f-45c6-8526-2c242576947c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.587469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 770/1529] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:57 2022] PUT /identity/v3/projects/13bdc95ac164408b9114a5cb69c9467a/users/729295e0893a4c989e4a1113691f7491/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:57.602105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea606ad1-8dd2-45b8-aa08-a6904e0833fa None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.602611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea606ad1-8dd2-45b8-aa08-a6904e0833fa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.602893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea606ad1-8dd2-45b8-aa08-a6904e0833fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.651267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ea606ad1-8dd2-45b8-aa08-a6904e0833fa None None] MFA Rules not processed for user `729295e0893a4c989e4a1113691f7491`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:57.653530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ea606ad1-8dd2-45b8-aa08-a6904e0833fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:57.663885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc7b8f-49de-4bba-9604-8df301ff11ce admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.772812 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ef982-6b15-4a85-8432-d6e01f3f053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.774361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef982-6b15-4a85-8432-d6e01f3f053d admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.775338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef982-6b15-4a85-8432-d6e01f3f053d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.775621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ef982-6b15-4a85-8432-d6e01f3f053d admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.777715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ef982-6b15-4a85-8432-d6e01f3f053d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.779833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ef982-6b15-4a85-8432-d6e01f3f053d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.820346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 771/1530] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:57.828050 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea606ad1-8dd2-45b8-aa08-a6904e0833fa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.846555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 760/1531] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:57 2022] POST /identity/v3/users => generated 292 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:57.850395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ef982-6b15-4a85-8432-d6e01f3f053d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.856243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea606ad1-8dd2-45b8-aa08-a6904e0833fa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.867458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ef982-6b15-4a85-8432-d6e01f3f053d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.869393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.870020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.870297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.870567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.871245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.875085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.884358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2720b0-eba5-460c-83f0-0bee2397898a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:57.886172 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2720b0-eba5-460c-83f0-0bee2397898a admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:57.886344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2720b0-eba5-460c-83f0-0bee2397898a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:57.886620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2720b0-eba5-460c-83f0-0bee2397898a admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:57.887303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2720b0-eba5-460c-83f0-0bee2397898a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:57.888898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2720b0-eba5-460c-83f0-0bee2397898a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:57.923689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 761/1532] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:57.927618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2720b0-eba5-460c-83f0-0bee2397898a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.941476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2720b0-eba5-460c-83f0-0bee2397898a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:57.942605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 772/1533] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:57 2022] POST /identity/v3/projects => generated 401 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:57.945935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:57.953342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:57.962132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:57.976471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:58.036387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9c153-20ef-4bf6-a99f-bd05624dc424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.037368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9c153-20ef-4bf6-a99f-bd05624dc424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.037818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9c153-20ef-4bf6-a99f-bd05624dc424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.038424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9c153-20ef-4bf6-a99f-bd05624dc424 admin admin] PATH_INFO: `/v3/projects/d069e25406b14777975d909a7499ec7a/users/ccfc7baf62824127ae1924f59c6e0347/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.058236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9c153-20ef-4bf6-a99f-bd05624dc424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d069e25406b14777975d909a7499ec7a, user_id=ccfc7baf62824127ae1924f59c6e0347, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.062232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9c153-20ef-4bf6-a99f-bd05624dc424 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.064843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed48195-adbc-4e54-8833-052737e3f364 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:58.071572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.072267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.072652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.073080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] PATH_INFO: `/v3/projects/0a3889a79a674ec98c66bbc7e3cec104` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.075506 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a3889a79a674ec98c66bbc7e3cec104)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.078081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.086947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 762/1534] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:57 2022] PUT /identity/v3/projects/d069e25406b14777975d909a7499ec7a/users/ccfc7baf62824127ae1924f59c6e0347/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:58.088212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9c153-20ef-4bf6-a99f-bd05624dc424 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.172362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9c153-20ef-4bf6-a99f-bd05624dc424 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:58.185227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a3889a79a674ec98c66bbc7e3cec104', 'request_id': 'req-0c502625-0709-4918-9f17-df782496eb9e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:58.203753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a3889a79a674ec98c66bbc7e3cec104', 'request_id': 'req-0c502625-0709-4918-9f17-df782496eb9e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:58.227603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a3889a79a674ec98c66bbc7e3cec104', 'request_id': 'req-0c502625-0709-4918-9f17-df782496eb9e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:58.252235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] The token cache is being invalidate because project 0a3889a79a674ec98c66bbc7e3cec104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:12:58.252481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:58.255212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:12:58.260800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 773/1535] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:12:57 2022] DELETE /identity/v3/projects/0a3889a79a674ec98c66bbc7e3cec104 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:58.261915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.271500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:58.286524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb529e-70e3-4812-8943-28be26db9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.287592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb529e-70e3-4812-8943-28be26db9a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.287950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb529e-70e3-4812-8943-28be26db9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.288235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb529e-70e3-4812-8943-28be26db9a00 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.288879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb529e-70e3-4812-8943-28be26db9a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.291692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb529e-70e3-4812-8943-28be26db9a00 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.320196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 763/1536] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:58 2022] POST /identity/v3/users => generated 307 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:58.322520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70d29f-8f50-4ef7-a5a1-8a7dfbda0d1e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.323083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70d29f-8f50-4ef7-a5a1-8a7dfbda0d1e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.323613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70d29f-8f50-4ef7-a5a1-8a7dfbda0d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.353724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0a70d29f-8f50-4ef7-a5a1-8a7dfbda0d1e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:58.355557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0a70d29f-8f50-4ef7-a5a1-8a7dfbda0d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:58.382438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c502625-0709-4918-9f17-df782496eb9e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:58.388811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7b013-2241-4ccf-931b-78f3a0318cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.389542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 764/1537] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:58.390504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b013-2241-4ccf-931b-78f3a0318cba admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.390783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b013-2241-4ccf-931b-78f3a0318cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.391601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7b013-2241-4ccf-931b-78f3a0318cba admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.392709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70d29f-8f50-4ef7-a5a1-8a7dfbda0d1e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.392818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7b013-2241-4ccf-931b-78f3a0318cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.395845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7b013-2241-4ccf-931b-78f3a0318cba admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.401707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0a70d29f-8f50-4ef7-a5a1-8a7dfbda0d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:58.411105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 774/1538] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:58.411950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24262c30-ea1f-4d09-94ba-2ffb124e653e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.412403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24262c30-ea1f-4d09-94ba-2ffb124e653e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.413345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24262c30-ea1f-4d09-94ba-2ffb124e653e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.430874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70d29f-8f50-4ef7-a5a1-8a7dfbda0d1e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:58.446383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28843d36-3173-4f50-8df9-0198a9b9177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.447672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28843d36-3173-4f50-8df9-0198a9b9177f admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.450192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28843d36-3173-4f50-8df9-0198a9b9177f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.450501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28843d36-3173-4f50-8df9-0198a9b9177f admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.451258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28843d36-3173-4f50-8df9-0198a9b9177f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.451789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-24262c30-ea1f-4d09-94ba-2ffb124e653e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:58.454651 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-24262c30-ea1f-4d09-94ba-2ffb124e653e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:58.456601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28843d36-3173-4f50-8df9-0198a9b9177f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.471643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 775/1539] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:58.472629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24262c30-ea1f-4d09-94ba-2ffb124e653e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.483802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 765/1540] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:58.491758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24262c30-ea1f-4d09-94ba-2ffb124e653e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:58.492037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28843d36-3173-4f50-8df9-0198a9b9177f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.501190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5d281-abe1-4bca-9049-62514788dca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.502219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5d281-abe1-4bca-9049-62514788dca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.502917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5d281-abe1-4bca-9049-62514788dca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.503441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5d281-abe1-4bca-9049-62514788dca8 admin admin] PATH_INFO: `/v3/projects/d069e25406b14777975d909a7499ec7a/users/ccfc7baf62824127ae1924f59c6e0347/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.506865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5d281-abe1-4bca-9049-62514788dca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d069e25406b14777975d909a7499ec7a, user_id=ccfc7baf62824127ae1924f59c6e0347, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.512460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5d281-abe1-4bca-9049-62514788dca8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.526198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 776/1541] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:58 2022] PUT /identity/v3/projects/d069e25406b14777975d909a7499ec7a/users/ccfc7baf62824127ae1924f59c6e0347/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:58.527290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5d281-abe1-4bca-9049-62514788dca8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.532070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28843d36-3173-4f50-8df9-0198a9b9177f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:58.556971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:58.578894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:58.674405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4df83-3efd-4a2f-8ec5-150e66424dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.675373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4df83-3efd-4a2f-8ec5-150e66424dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.675980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4df83-3efd-4a2f-8ec5-150e66424dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.676515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4df83-3efd-4a2f-8ec5-150e66424dd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.679013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4df83-3efd-4a2f-8ec5-150e66424dd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:12:58.681647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4df83-3efd-4a2f-8ec5-150e66424dd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.682456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5d281-abe1-4bca-9049-62514788dca8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:58.685911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4df83-3efd-4a2f-8ec5-150e66424dd5 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.688748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa5a77-71c5-4600-8a68-ac90992b3a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.689414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa5a77-71c5-4600-8a68-ac90992b3a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.689696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa5a77-71c5-4600-8a68-ac90992b3a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.689970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa5a77-71c5-4600-8a68-ac90992b3a47 admin admin] PATH_INFO: `/v3/projects/3cff8343a10d4a3c810bf2e587de5bb9/users/b71bad03bc6941ce9f7629c98cd1954f/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.708174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 766/1542] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:12:58.709857 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa5a77-71c5-4600-8a68-ac90992b3a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cff8343a10d4a3c810bf2e587de5bb9, user_id=b71bad03bc6941ce9f7629c98cd1954f, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.710189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da066729-7586-4e78-9d35-ea95239ca22c None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.710645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da066729-7586-4e78-9d35-ea95239ca22c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.711086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da066729-7586-4e78-9d35-ea95239ca22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.712748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa5a77-71c5-4600-8a68-ac90992b3a47 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.742418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 777/1543] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:58 2022] PUT /identity/v3/projects/3cff8343a10d4a3c810bf2e587de5bb9/users/b71bad03bc6941ce9f7629c98cd1954f/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:58.742418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa5a77-71c5-4600-8a68-ac90992b3a47 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.795029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-da066729-7586-4e78-9d35-ea95239ca22c None None] MFA Rules not processed for user `ccfc7baf62824127ae1924f59c6e0347`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:58.800289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-da066729-7586-4e78-9d35-ea95239ca22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:58.823658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa5a77-71c5-4600-8a68-ac90992b3a47 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:58.830824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584d3c7-a362-4d47-a577-e9b5146dd29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:58.831707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584d3c7-a362-4d47-a577-e9b5146dd29a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:58.832087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584d3c7-a362-4d47-a577-e9b5146dd29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:58.832462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584d3c7-a362-4d47-a577-e9b5146dd29a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:58.833196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584d3c7-a362-4d47-a577-e9b5146dd29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:58.835516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584d3c7-a362-4d47-a577-e9b5146dd29a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:58.897767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 778/1544] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:12:58 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:58.900295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584d3c7-a362-4d47-a577-e9b5146dd29a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.948958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 767/1545] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:58.949936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da066729-7586-4e78-9d35-ea95239ca22c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:58.962657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da066729-7586-4e78-9d35-ea95239ca22c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.053400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584d3c7-a362-4d47-a577-e9b5146dd29a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.060823 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cbd0e-2359-4bb1-94dc-37dbd4d883fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.066397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cbd0e-2359-4bb1-94dc-37dbd4d883fa admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.066397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cbd0e-2359-4bb1-94dc-37dbd4d883fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.066397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cbd0e-2359-4bb1-94dc-37dbd4d883fa admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.066397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900cbd0e-2359-4bb1-94dc-37dbd4d883fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:59.069963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900cbd0e-2359-4bb1-94dc-37dbd4d883fa admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:59.074615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.075292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.075583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.075869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.076540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:59.078752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:59.103489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 779/1546] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:59.105060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:59.107088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:59.122448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 768/1547] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:12:58 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:12:59.124114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cbd0e-2359-4bb1-94dc-37dbd4d883fa admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:59.142016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cbd0e-2359-4bb1-94dc-37dbd4d883fa admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.149867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff97f-0be0-49d3-b23e-4146ccc86a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.150048 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff97f-0be0-49d3-b23e-4146ccc86a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.150432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff97f-0be0-49d3-b23e-4146ccc86a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.150678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfff97f-0be0-49d3-b23e-4146ccc86a74 admin admin] PATH_INFO: `/v3/projects/3cff8343a10d4a3c810bf2e587de5bb9/users/b71bad03bc6941ce9f7629c98cd1954f/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.152826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfff97f-0be0-49d3-b23e-4146ccc86a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cff8343a10d4a3c810bf2e587de5bb9, user_id=b71bad03bc6941ce9f7629c98cd1954f, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:59.157113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfff97f-0be0-49d3-b23e-4146ccc86a74 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:59.162911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:59.184662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 769/1548] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:59 2022] PUT /identity/v3/projects/3cff8343a10d4a3c810bf2e587de5bb9/users/b71bad03bc6941ce9f7629c98cd1954f/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:59.184662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff97f-0be0-49d3-b23e-4146ccc86a74 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:59.198322 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:12:59.242135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974f1ca-afd3-4447-8a6a-82a058a707d0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.315411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986363b-55f5-4b63-9f82-05cf8a9cebd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.317761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986363b-55f5-4b63-9f82-05cf8a9cebd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.318313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986363b-55f5-4b63-9f82-05cf8a9cebd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.319240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986363b-55f5-4b63-9f82-05cf8a9cebd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.324513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfff97f-0be0-49d3-b23e-4146ccc86a74 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.340918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5836a-c5eb-4faa-acbe-5270af3b0488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.341637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5836a-c5eb-4faa-acbe-5270af3b0488 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.341916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5836a-c5eb-4faa-acbe-5270af3b0488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.342188 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5836a-c5eb-4faa-acbe-5270af3b0488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.344710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5836a-c5eb-4faa-acbe-5270af3b0488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:59.347315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5836a-c5eb-4faa-acbe-5270af3b0488 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:59.369961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 770/1549] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:59.375917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e64e2-a1b4-414e-bf22-249892bfb007 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.376201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e64e2-a1b4-414e-bf22-249892bfb007 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.376475 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e64e2-a1b4-414e-bf22-249892bfb007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.449373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-492e64e2-a1b4-414e-bf22-249892bfb007 None None] MFA Rules not processed for user `b71bad03bc6941ce9f7629c98cd1954f`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:59.452360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-492e64e2-a1b4-414e-bf22-249892bfb007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:59.473530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986363b-55f5-4b63-9f82-05cf8a9cebd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:59.475328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5986363b-55f5-4b63-9f82-05cf8a9cebd0 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:59.499143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 780/1550] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:59.501114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986363b-55f5-4b63-9f82-05cf8a9cebd0 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:59.529647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986363b-55f5-4b63-9f82-05cf8a9cebd0 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.540409 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.542059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.542840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.545745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.547176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:59.551087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:59.584576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 771/1551] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:59 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:12:59.587682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 781/1552] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:12:59.591532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e64e2-a1b4-414e-bf22-249892bfb007 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:59.613717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e64e2-a1b4-414e-bf22-249892bfb007 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.627714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa67b1-59f8-48b6-af36-9e2a34d2e161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.628671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa67b1-59f8-48b6-af36-9e2a34d2e161 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.629057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa67b1-59f8-48b6-af36-9e2a34d2e161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.629863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa67b1-59f8-48b6-af36-9e2a34d2e161 admin admin] PATH_INFO: `/v3/projects/fdf2c39dfd564ac7817c7e8ed1ca6a99/users/752b7dd8c41f4fe1841dccf30157ed8d/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.637237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:59.641435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:59.670109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d83e36-6bf6-4a71-b64d-46e67f1dd945 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.675203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b075-e70d-47c2-8185-65344dab6943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.675945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b075-e70d-47c2-8185-65344dab6943 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.676239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b075-e70d-47c2-8185-65344dab6943 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.676522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b075-e70d-47c2-8185-65344dab6943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.694017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa67b1-59f8-48b6-af36-9e2a34d2e161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf2c39dfd564ac7817c7e8ed1ca6a99, user_id=752b7dd8c41f4fe1841dccf30157ed8d, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:59.698593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa67b1-59f8-48b6-af36-9e2a34d2e161 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:59.700227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3b075-e70d-47c2-8185-65344dab6943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:12:59.704108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3b075-e70d-47c2-8185-65344dab6943 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:12:59.755203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 782/1553] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:59 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:12:59.755464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 772/1554] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:12:59 2022] PUT /identity/v3/projects/fdf2c39dfd564ac7817c7e8ed1ca6a99/users/752b7dd8c41f4fe1841dccf30157ed8d/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:12:59.761272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b075-e70d-47c2-8185-65344dab6943 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:12:59.773201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f590bd-9a4b-4e3c-8dd0-d6f5923cfd2c None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.773642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f590bd-9a4b-4e3c-8dd0-d6f5923cfd2c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.773932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f590bd-9a4b-4e3c-8dd0-d6f5923cfd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.796781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b075-e70d-47c2-8185-65344dab6943 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:12:59.874546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-13f590bd-9a4b-4e3c-8dd0-d6f5923cfd2c None None] MFA Rules not processed for user `752b7dd8c41f4fe1841dccf30157ed8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:12:59.876403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-13f590bd-9a4b-4e3c-8dd0-d6f5923cfd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:59.963782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:12:59.964452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:12:59.964743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:12:59.965077 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:12:59.966919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:12:59.981602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:00.096818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:00.108646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 773/1555] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:12:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:00.109013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:00.111648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f590bd-9a4b-4e3c-8dd0-d6f5923cfd2c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:00.114081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:00.156713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 783/1556] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:12:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:00.158409 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:00.160634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f590bd-9a4b-4e3c-8dd0-d6f5923cfd2c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:00.167615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc486c9-fe21-43b8-94b9-e2173de45ec6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:00.168580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc486c9-fe21-43b8-94b9-e2173de45ec6 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.169452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc486c9-fe21-43b8-94b9-e2173de45ec6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.170032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc486c9-fe21-43b8-94b9-e2173de45ec6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.179522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9cc486c9-fe21-43b8-94b9-e2173de45ec6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:00.301890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc486c9-fe21-43b8-94b9-e2173de45ec6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:00.308994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c14fc-5ab6-4c22-97bb-9ebbb066c6b2 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:00.309237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc486c9-fe21-43b8-94b9-e2173de45ec6 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:00.314363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bca9c-f1f4-4eb4-aa81-b425b21ae3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:00.317601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bca9c-f1f4-4eb4-aa81-b425b21ae3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.317940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bca9c-f1f4-4eb4-aa81-b425b21ae3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.318485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bca9c-f1f4-4eb4-aa81-b425b21ae3c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.318887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234bca9c-f1f4-4eb4-aa81-b425b21ae3c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:00.326248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234bca9c-f1f4-4eb4-aa81-b425b21ae3c8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:00.330438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 774/1557] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:00.332545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a638f8-b7f4-47ef-935a-c945c08c77ee None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.332946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a638f8-b7f4-47ef-935a-c945c08c77ee None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.333332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a638f8-b7f4-47ef-935a-c945c08c77ee None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.335495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 775/1558] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:13:00.415004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 784/1559] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:00 2022] POST /identity/v3/projects => generated 387 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:00.416215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a638f8-b7f4-47ef-935a-c945c08c77ee None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:00.458404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efbd84-d19c-4fa4-965e-be6260458ff8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.459098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efbd84-d19c-4fa4-965e-be6260458ff8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.459098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efbd84-d19c-4fa4-965e-be6260458ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.493123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-75efbd84-d19c-4fa4-965e-be6260458ff8 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:00.495926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-75efbd84-d19c-4fa4-965e-be6260458ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:00.611029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a638f8-b7f4-47ef-935a-c945c08c77ee None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:00.621261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957f65c-f0cb-4965-8b69-d65d9f97eb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:00.623912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957f65c-f0cb-4965-8b69-d65d9f97eb10 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.624221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957f65c-f0cb-4965-8b69-d65d9f97eb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.624493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957f65c-f0cb-4965-8b69-d65d9f97eb10 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.626367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957f65c-f0cb-4965-8b69-d65d9f97eb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:00.631642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957f65c-f0cb-4965-8b69-d65d9f97eb10 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:00.653618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 785/1560] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:13:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:00.681428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efbd84-d19c-4fa4-965e-be6260458ff8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:00.699434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 776/1561] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:00 2022] POST /identity/v3/users => generated 300 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:00.710305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efbd84-d19c-4fa4-965e-be6260458ff8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:00.721749 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:00.722479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.722873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.723336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.724261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:00.726244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:00.747656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957f65c-f0cb-4965-8b69-d65d9f97eb10 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:00.776433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 786/1562] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:00.777632 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:00.788207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:00.797401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957f65c-f0cb-4965-8b69-d65d9f97eb10 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:00.828598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:00.830617 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.831628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.832011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] PATH_INFO: `/v3/projects/42b7bf11266f4645a9d613dfab3f13cc/users/622cfa7d30354bbc800979e6b261e548/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.853261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b7bf11266f4645a9d613dfab3f13cc, user_id=622cfa7d30354bbc800979e6b261e548, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:00.853458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82396b00-69d6-4328-84dc-c5790adb36b3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:00.856765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:00.861167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ed842-28e0-4339-aa89-97ebec9bd4b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:00.861839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ed842-28e0-4339-aa89-97ebec9bd4b1 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.862175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ed842-28e0-4339-aa89-97ebec9bd4b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.862616 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ed842-28e0-4339-aa89-97ebec9bd4b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.874413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ed842-28e0-4339-aa89-97ebec9bd4b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:00.875979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ed842-28e0-4339-aa89-97ebec9bd4b1 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:00.879767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 777/1563] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:00 2022] PUT /identity/v3/projects/42b7bf11266f4645a9d613dfab3f13cc/users/622cfa7d30354bbc800979e6b261e548/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:00.880885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:00.882928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:00.891525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 787/1564] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:00.892508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ed842-28e0-4339-aa89-97ebec9bd4b1 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:00.899239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:00.904594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ed842-28e0-4339-aa89-97ebec9bd4b1 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:00.976204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da751e-1df4-47da-b947-22dd420ba1de admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:00.982158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:00.984387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:00.984803 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:00.985182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] PATH_INFO: `/v3/users/5616c0f1b99c4a6bbe0adf0ea0e130f8` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:00.987001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5616c0f1b99c4a6bbe0adf0ea0e130f8)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:00.990131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:00.996943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:01.109977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5616c0f1b99c4a6bbe0adf0ea0e130f8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:01.112039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:01.113106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:01.113106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:01.114097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:01.115943 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:01.122867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:01.141677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5616c0f1b99c4a6bbe0adf0ea0e130f8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:01.143066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5616c0f1b99c4a6bbe0adf0ea0e130f8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:01.155365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 788/1565] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:01.158462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5616c0f1b99c4a6bbe0adf0ea0e130f8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:01.159450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:01.165768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:01.170321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d79fb76-c7f5-489c-afdb-7b7bd4b86396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5616c0f1b99c4a6bbe0adf0ea0e130f8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:01.187425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 778/1566] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:00 2022] DELETE /identity/v3/users/5616c0f1b99c4a6bbe0adf0ea0e130f8 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:01.187912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a93c10-a14f-4d50-af0c-26a0ae6cef48 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:01.191740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb81fbd-6336-44a8-ace4-349352c6dc86 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:01.192392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb81fbd-6336-44a8-ace4-349352c6dc86 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:01.192978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb81fbd-6336-44a8-ace4-349352c6dc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:01.221551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-bbb81fbd-6336-44a8-ace4-349352c6dc86 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:01.223286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-bbb81fbd-6336-44a8-ace4-349352c6dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:01.278563 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864d0f31-1364-432c-9b6a-d57777f9196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:01.279427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d0f31-1364-432c-9b6a-d57777f9196e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:01.282365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d0f31-1364-432c-9b6a-d57777f9196e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:01.282855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864d0f31-1364-432c-9b6a-d57777f9196e admin admin] PATH_INFO: `/v3/projects/42b7bf11266f4645a9d613dfab3f13cc/users/622cfa7d30354bbc800979e6b261e548/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:01.285375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d0f31-1364-432c-9b6a-d57777f9196e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b7bf11266f4645a9d613dfab3f13cc, user_id=622cfa7d30354bbc800979e6b261e548, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:01.288725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864d0f31-1364-432c-9b6a-d57777f9196e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:01.301543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 789/1567] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:01 2022] PUT /identity/v3/projects/42b7bf11266f4645a9d613dfab3f13cc/users/622cfa7d30354bbc800979e6b261e548/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:01.304339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 779/1568] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:01.313589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cd209-6e71-446f-b281-4a0a0b00d96c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:01.313895 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cd209-6e71-446f-b281-4a0a0b00d96c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:01.314843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cd209-6e71-446f-b281-4a0a0b00d96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:01.347512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d69cd209-6e71-446f-b281-4a0a0b00d96c None None] MFA Rules not processed for user `622cfa7d30354bbc800979e6b261e548`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:01.348928 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d69cd209-6e71-446f-b281-4a0a0b00d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:01.382190 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb81fbd-6336-44a8-ace4-349352c6dc86 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:01.386252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-bbb81fbd-6336-44a8-ace4-349352c6dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:01.466463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb81fbd-6336-44a8-ace4-349352c6dc86 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:01.532642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:01.557026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:01.558016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:01.560613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:01.561030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:01.584106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 790/1569] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:01.585607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cd209-6e71-446f-b281-4a0a0b00d96c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:01.590547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d69cd209-6e71-446f-b281-4a0a0b00d96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:01.664764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:01.800572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:01.801750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cd209-6e71-446f-b281-4a0a0b00d96c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:01.806402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:01.813046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00826abf-6ec6-4e30-be7e-1faf4e3ad660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:01.813756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826abf-6ec6-4e30-be7e-1faf4e3ad660 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:01.814115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826abf-6ec6-4e30-be7e-1faf4e3ad660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:01.815238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826abf-6ec6-4e30-be7e-1faf4e3ad660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:01.831026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00826abf-6ec6-4e30-be7e-1faf4e3ad660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:01.832782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00826abf-6ec6-4e30-be7e-1faf4e3ad660 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:01.833189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 780/1570] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:01.850951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 791/1571] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:02.756755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:02.758226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:02.780234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109a1e9-9d92-4394-b9fd-93995fdea11b service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:02.807084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:02.807837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:02.808186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:02.808571 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] PATH_INFO: `/v3/projects/bf3c13ae2890450ea18850ac2eecde74` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:02.809821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf3c13ae2890450ea18850ac2eecde74)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:02.812654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:02.955084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf3c13ae2890450ea18850ac2eecde74', 'request_id': 'req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:02.976815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf3c13ae2890450ea18850ac2eecde74', 'request_id': 'req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.001597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf3c13ae2890450ea18850ac2eecde74', 'request_id': 'req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.036104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] The token cache is being invalidate because project bf3c13ae2890450ea18850ac2eecde74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:03.040253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.041882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-024e2f74-c760-4f1e-a0a5-c2d64de4dddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.046672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 781/1572] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:02 2022] DELETE /identity/v3/projects/bf3c13ae2890450ea18850ac2eecde74 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:03.054094 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35552-af53-4543-a6f9-ba4db5aa5152 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.054987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35552-af53-4543-a6f9-ba4db5aa5152 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.055502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35552-af53-4543-a6f9-ba4db5aa5152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.101760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e3d35552-af53-4543-a6f9-ba4db5aa5152 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:03.103079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e3d35552-af53-4543-a6f9-ba4db5aa5152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:03.208401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 792/1573] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:03.215094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4038-d946-4b5c-8c84-c81590e3a057 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.215404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4038-d946-4b5c-8c84-c81590e3a057 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.215687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4038-d946-4b5c-8c84-c81590e3a057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.239454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-354c4038-d946-4b5c-8c84-c81590e3a057 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:03.240801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-354c4038-d946-4b5c-8c84-c81590e3a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:03.256001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 782/1574] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:03.259594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d35552-af53-4543-a6f9-ba4db5aa5152 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:03.284683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d35552-af53-4543-a6f9-ba4db5aa5152 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:03.288100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c4038-d946-4b5c-8c84-c81590e3a057 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:03.290244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-354c4038-d946-4b5c-8c84-c81590e3a057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:03.291380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:03.292013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.292477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.292760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.293427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:03.293738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:03.296121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:03.306883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c4038-d946-4b5c-8c84-c81590e3a057 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:03.312449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:03.313224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.313601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.314157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] PATH_INFO: `/v3/users/752b7dd8c41f4fe1841dccf30157ed8d` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.315575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 793/1575] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:03.319328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:03.320571 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=752b7dd8c41f4fe1841dccf30157ed8d)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:03.322793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:03.331995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5cde-b714-4815-bc49-1a9266515abd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:03.337668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d159750-6e67-4560-a094-5b2508b46b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:03.338387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d159750-6e67-4560-a094-5b2508b46b4f admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.338776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d159750-6e67-4560-a094-5b2508b46b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.339199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d159750-6e67-4560-a094-5b2508b46b4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.340219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d159750-6e67-4560-a094-5b2508b46b4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:03.342460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d159750-6e67-4560-a094-5b2508b46b4f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:03.376417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 794/1576] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:03 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:03.380544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d159750-6e67-4560-a094-5b2508b46b4f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:03.395755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752b7dd8c41f4fe1841dccf30157ed8d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.409664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:03.413431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752b7dd8c41f4fe1841dccf30157ed8d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.414930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752b7dd8c41f4fe1841dccf30157ed8d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.424364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752b7dd8c41f4fe1841dccf30157ed8d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.434627 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-078aac9d-6656-4934-8e07-e4fa910d2a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752b7dd8c41f4fe1841dccf30157ed8d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:03.446578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 783/1577] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:03 2022] DELETE /identity/v3/users/752b7dd8c41f4fe1841dccf30157ed8d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:03.451966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f500a-faa2-469e-b5e0-c7365a2d223b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.452464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f500a-faa2-469e-b5e0-c7365a2d223b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.452750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f500a-faa2-469e-b5e0-c7365a2d223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.461799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d159750-6e67-4560-a094-5b2508b46b4f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:03.485799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5e5f500a-faa2-469e-b5e0-c7365a2d223b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:03.487307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5e5f500a-faa2-469e-b5e0-c7365a2d223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:03.525131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:03.559209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:03.559989 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.560388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.560766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.562549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:03.563953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 784/1578] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:03.564704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:03.575096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f500a-faa2-469e-b5e0-c7365a2d223b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:03.577508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5e5f500a-faa2-469e-b5e0-c7365a2d223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:03.596477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 795/1579] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:03 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:03.600694 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:03.602933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:03.620291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a14fc-66dd-4bbf-9d7c-fd4977f9ed88 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:03.625483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39054809-b00c-411e-a4fa-cbb14d4c53f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:03.626118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39054809-b00c-411e-a4fa-cbb14d4c53f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.626400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39054809-b00c-411e-a4fa-cbb14d4c53f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.626677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39054809-b00c-411e-a4fa-cbb14d4c53f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.627375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39054809-b00c-411e-a4fa-cbb14d4c53f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:03.629182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39054809-b00c-411e-a4fa-cbb14d4c53f0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:03.644866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 796/1580] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:03.648025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39054809-b00c-411e-a4fa-cbb14d4c53f0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:03.662965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39054809-b00c-411e-a4fa-cbb14d4c53f0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:03.671046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfb7a3-edf1-4ec5-b6a8-199b28b4b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:03.673507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfb7a3-edf1-4ec5-b6a8-199b28b4b21d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.673878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfb7a3-edf1-4ec5-b6a8-199b28b4b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.674230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfb7a3-edf1-4ec5-b6a8-199b28b4b21d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.675020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfb7a3-edf1-4ec5-b6a8-199b28b4b21d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:03.677027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfb7a3-edf1-4ec5-b6a8-199b28b4b21d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:03.690691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 797/1581] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:13:03.695487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfb7a3-edf1-4ec5-b6a8-199b28b4b21d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:03.707793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f500a-faa2-469e-b5e0-c7365a2d223b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:03.708587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfb7a3-edf1-4ec5-b6a8-199b28b4b21d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:03.712250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d7f55-299e-4840-8bb6-33cd43c1cc14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:03.712879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7f55-299e-4840-8bb6-33cd43c1cc14 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.713166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7f55-299e-4840-8bb6-33cd43c1cc14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.713438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d7f55-299e-4840-8bb6-33cd43c1cc14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.718345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:03.719205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.719633 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.720115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] PATH_INFO: `/v3/projects/59dd0055a7684dc2a7e0c94540d10f48/users/1c5f2f1df3e7472e8fa90a2bd8acbe56/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.725315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d7f55-299e-4840-8bb6-33cd43c1cc14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:03.726788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d7f55-299e-4840-8bb6-33cd43c1cc14 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:03.743244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 785/1582] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:03.745610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59dd0055a7684dc2a7e0c94540d10f48, user_id=1c5f2f1df3e7472e8fa90a2bd8acbe56, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:03.748699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:03.778251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6afbc3-af42-4ae6-aa77-e9eaa019560a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:03.778543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6afbc3-af42-4ae6-aa77-e9eaa019560a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:03.778827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6afbc3-af42-4ae6-aa77-e9eaa019560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:03.783285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 798/1583] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:03 2022] PUT /identity/v3/projects/59dd0055a7684dc2a7e0c94540d10f48/users/1c5f2f1df3e7472e8fa90a2bd8acbe56/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:03.856470 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1b6afbc3-af42-4ae6-aa77-e9eaa019560a None None] MFA Rules not processed for user `1c5f2f1df3e7472e8fa90a2bd8acbe56`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:03.857751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1b6afbc3-af42-4ae6-aa77-e9eaa019560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:04.024962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 786/1584] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:04.037585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:04.040700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:04.138514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3549e7-fc1b-4605-b7b6-5658260758e2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:04.144128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f30708-9f46-46c5-9b83-ed16e8b67067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:04.145082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f30708-9f46-46c5-9b83-ed16e8b67067 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:04.145385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f30708-9f46-46c5-9b83-ed16e8b67067 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:04.145733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f30708-9f46-46c5-9b83-ed16e8b67067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:04.158176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f30708-9f46-46c5-9b83-ed16e8b67067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:04.159603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f30708-9f46-46c5-9b83-ed16e8b67067 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:04.187212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 799/1585] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:04.416364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6afbc3-af42-4ae6-aa77-e9eaa019560a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:04.418104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1b6afbc3-af42-4ae6-aa77-e9eaa019560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:04.442474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:04.550443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6afbc3-af42-4ae6-aa77-e9eaa019560a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:04.556908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:04.557822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:04.558104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:04.558383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] PATH_INFO: `/v3/projects/fdf2c39dfd564ac7817c7e8ed1ca6a99` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:04.559626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf2c39dfd564ac7817c7e8ed1ca6a99)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:04.562179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:04.656724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf2c39dfd564ac7817c7e8ed1ca6a99', 'request_id': 'req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.665064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf2c39dfd564ac7817c7e8ed1ca6a99', 'request_id': 'req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.674685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf2c39dfd564ac7817c7e8ed1ca6a99', 'request_id': 'req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.688881 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] The token cache is being invalidate because project fdf2c39dfd564ac7817c7e8ed1ca6a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:04.689187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.689852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.692001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 787/1586] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:04 2022] DELETE /identity/v3/projects/fdf2c39dfd564ac7817c7e8ed1ca6a99 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:04.696441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f30708-9f46-46c5-9b83-ed16e8b67067 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:04.708436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f30708-9f46-46c5-9b83-ed16e8b67067 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:04.769547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:04.770186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:04.770472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:04.770752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] PATH_INFO: `/v3/users/622cfa7d30354bbc800979e6b261e548` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:04.771910 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] RBAC: Authorizing `identity:delete_user(user_id=622cfa7d30354bbc800979e6b261e548)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:04.773868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:04.797975 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:04.825085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '622cfa7d30354bbc800979e6b261e548'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.839248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '622cfa7d30354bbc800979e6b261e548'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.840037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '622cfa7d30354bbc800979e6b261e548'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.850099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '622cfa7d30354bbc800979e6b261e548'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.850966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aba40734-01bb-426f-af8a-03e29405f69d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '622cfa7d30354bbc800979e6b261e548'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:04.865836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 800/1587] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:04 2022] DELETE /identity/v3/users/622cfa7d30354bbc800979e6b261e548 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:05.190636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:05.192214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:05.206388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576c4f-bd1b-4d49-9fe4-1657b12a8f96 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:05.271302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:05.271957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.272246 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.273816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] PATH_INFO: `/v3/projects/42b7bf11266f4645a9d613dfab3f13cc` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.275010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b7bf11266f4645a9d613dfab3f13cc)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:05.277609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:05.329527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:05.338284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b7bf11266f4645a9d613dfab3f13cc', 'request_id': 'req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:05.346485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b7bf11266f4645a9d613dfab3f13cc', 'request_id': 'req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:05.356032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b7bf11266f4645a9d613dfab3f13cc', 'request_id': 'req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:05.370935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] The token cache is being invalidate because project 42b7bf11266f4645a9d613dfab3f13cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:05.371257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:05.371947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-fc2326e2-37a2-424d-98c7-733fcdf9b6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:05.374048 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 788/1588] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:05 2022] DELETE /identity/v3/projects/42b7bf11266f4645a9d613dfab3f13cc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:05.378939 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a2cf-2061-4d1e-97e7-d789408f06d3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.379425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a2cf-2061-4d1e-97e7-d789408f06d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.379499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a2cf-2061-4d1e-97e7-d789408f06d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.400970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-aca8a2cf-2061-4d1e-97e7-d789408f06d3 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:05.402176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-aca8a2cf-2061-4d1e-97e7-d789408f06d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:05.464216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 801/1589] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:05.471850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de472-35bf-419f-a154-b1656e1d45d1 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.472149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de472-35bf-419f-a154-b1656e1d45d1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.472435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076de472-35bf-419f-a154-b1656e1d45d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.492690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-076de472-35bf-419f-a154-b1656e1d45d1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:05.493905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-076de472-35bf-419f-a154-b1656e1d45d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:05.503228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 789/1590] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:05.507755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8a2cf-2061-4d1e-97e7-d789408f06d3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:05.519288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8a2cf-2061-4d1e-97e7-d789408f06d3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:05.523485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:05.524454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.524744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.525019 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.525670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:05.526009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:05.527631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:05.539153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 802/1591] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:13:05.543555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076de472-35bf-419f-a154-b1656e1d45d1 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:05.553755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076de472-35bf-419f-a154-b1656e1d45d1 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:05.558332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e462f7-f7d6-471c-8068-5ae57d1da3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:05.558934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e462f7-f7d6-471c-8068-5ae57d1da3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.559253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e462f7-f7d6-471c-8068-5ae57d1da3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.559534 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e462f7-f7d6-471c-8068-5ae57d1da3d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.560173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e462f7-f7d6-471c-8068-5ae57d1da3d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:05.562713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e462f7-f7d6-471c-8068-5ae57d1da3d0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:05.592115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 790/1592] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:05 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:05.596352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:05.606598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46dcde0-8e1d-440a-bf14-fa6f74a2c02c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:05.666033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da575c-f8b1-41cc-b813-d4245451f21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:05.666634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da575c-f8b1-41cc-b813-d4245451f21f admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.666917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da575c-f8b1-41cc-b813-d4245451f21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.667227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da575c-f8b1-41cc-b813-d4245451f21f admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.667879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da575c-f8b1-41cc-b813-d4245451f21f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:05.669858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da575c-f8b1-41cc-b813-d4245451f21f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:05.699464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 803/1593] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:05 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:05.700745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e462f7-f7d6-471c-8068-5ae57d1da3d0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:05.718243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e462f7-f7d6-471c-8068-5ae57d1da3d0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:05.724709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503d89e-3004-4994-85ed-2160bd4370b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:05.725341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d89e-3004-4994-85ed-2160bd4370b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.725620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d89e-3004-4994-85ed-2160bd4370b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.725898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503d89e-3004-4994-85ed-2160bd4370b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.726598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503d89e-3004-4994-85ed-2160bd4370b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:05.728357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503d89e-3004-4994-85ed-2160bd4370b8 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:05.739621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 791/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:05.743712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da575c-f8b1-41cc-b813-d4245451f21f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:05.755529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da575c-f8b1-41cc-b813-d4245451f21f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:05.760926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77801f7a-a128-4a36-aa8e-84fa825b14b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:05.761668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801f7a-a128-4a36-aa8e-84fa825b14b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.762029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801f7a-a128-4a36-aa8e-84fa825b14b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.762319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77801f7a-a128-4a36-aa8e-84fa825b14b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.763038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77801f7a-a128-4a36-aa8e-84fa825b14b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:05.765000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77801f7a-a128-4a36-aa8e-84fa825b14b2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:05.781484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 804/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:05.785690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503d89e-3004-4994-85ed-2160bd4370b8 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:05.796277 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503d89e-3004-4994-85ed-2160bd4370b8 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:05.800817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:05.801466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.801749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.802028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] PATH_INFO: `/v3/projects/42e8ff8265f24a248d5e2f69796e803c/users/247fa7c2e727444c9bb969bd6991b7e5/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.816845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e8ff8265f24a248d5e2f69796e803c, user_id=247fa7c2e727444c9bb969bd6991b7e5, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:05.819528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:05.831978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 792/1596] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:05 2022] PUT /identity/v3/projects/42e8ff8265f24a248d5e2f69796e803c/users/247fa7c2e727444c9bb969bd6991b7e5/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:05.837014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689da618-c03e-418f-a56b-7fbfecac0415 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.837304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689da618-c03e-418f-a56b-7fbfecac0415 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.837585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689da618-c03e-418f-a56b-7fbfecac0415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:05.851566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:05.857703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-689da618-c03e-418f-a56b-7fbfecac0415 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:05.860022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-689da618-c03e-418f-a56b-7fbfecac0415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:05.917093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 805/1597] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:05.927393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:05.929084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:05.987960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554cff3-a2be-4d18-b641-a853f3adc400 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:05.991404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aef530-5750-4bcd-bb51-b4e66fcfa76a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:05.992008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aef530-5750-4bcd-bb51-b4e66fcfa76a service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:05.992314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aef530-5750-4bcd-bb51-b4e66fcfa76a service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:05.992584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aef530-5750-4bcd-bb51-b4e66fcfa76a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:06.002283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aef530-5750-4bcd-bb51-b4e66fcfa76a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:06.003975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aef530-5750-4bcd-bb51-b4e66fcfa76a service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:06.013611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 793/1598] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:06.737077 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a28eb-a7ef-43b9-bdc2-5fdb9340da6e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:06.738282 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a28eb-a7ef-43b9-bdc2-5fdb9340da6e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:06.738793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a28eb-a7ef-43b9-bdc2-5fdb9340da6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:06.794147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5d9a28eb-a7ef-43b9-bdc2-5fdb9340da6e None None] MFA Rules not processed for user `247fa7c2e727444c9bb969bd6991b7e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:06.797622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5d9a28eb-a7ef-43b9-bdc2-5fdb9340da6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:06.970247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 806/1599] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:06.970786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aef530-5750-4bcd-bb51-b4e66fcfa76a service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:06.985210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aef530-5750-4bcd-bb51-b4e66fcfa76a service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:06.995303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd21539-659e-4b46-99b3-5965729b3c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:07.003106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd21539-659e-4b46-99b3-5965729b3c6a admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:07.003468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd21539-659e-4b46-99b3-5965729b3c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:07.003758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd21539-659e-4b46-99b3-5965729b3c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:07.004477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd21539-659e-4b46-99b3-5965729b3c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:07.010895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd21539-659e-4b46-99b3-5965729b3c6a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:07.055752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 794/1600] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:06 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:07.059744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a28eb-a7ef-43b9-bdc2-5fdb9340da6e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:07.079230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a28eb-a7ef-43b9-bdc2-5fdb9340da6e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:07.185892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f73aa-1d04-4ac5-91e1-d6e1451fb389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:07.187359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f73aa-1d04-4ac5-91e1-d6e1451fb389 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:07.187765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f73aa-1d04-4ac5-91e1-d6e1451fb389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:07.188052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f73aa-1d04-4ac5-91e1-d6e1451fb389 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:07.189390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f73aa-1d04-4ac5-91e1-d6e1451fb389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:07.192828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f73aa-1d04-4ac5-91e1-d6e1451fb389 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:07.227505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 807/1601] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:07 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:07.231256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd21539-659e-4b46-99b3-5965729b3c6a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:07.252901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd21539-659e-4b46-99b3-5965729b3c6a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:07.262525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee3a8d-63d3-4b0e-8510-e897517a52e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:07.263257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3a8d-63d3-4b0e-8510-e897517a52e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:07.263541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3a8d-63d3-4b0e-8510-e897517a52e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:07.263822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3a8d-63d3-4b0e-8510-e897517a52e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:07.264515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee3a8d-63d3-4b0e-8510-e897517a52e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:07.266173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee3a8d-63d3-4b0e-8510-e897517a52e4 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:07.278641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 795/1602] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:07.283438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f73aa-1d04-4ac5-91e1-d6e1451fb389 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:07.298588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f73aa-1d04-4ac5-91e1-d6e1451fb389 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:07.305604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648940-ce6c-4b86-bb0d-4acc425886bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:07.306291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648940-ce6c-4b86-bb0d-4acc425886bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:07.306578 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648940-ce6c-4b86-bb0d-4acc425886bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:07.306857 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb648940-ce6c-4b86-bb0d-4acc425886bd admin admin] PATH_INFO: `/v3/projects/cd4daec70cc246f2bde3c88398ad6488/users/fbec55028f1d40e09951cb1a76999f28/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:07.329851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb648940-ce6c-4b86-bb0d-4acc425886bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd4daec70cc246f2bde3c88398ad6488, user_id=fbec55028f1d40e09951cb1a76999f28, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:07.332851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb648940-ce6c-4b86-bb0d-4acc425886bd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:07.350617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 808/1603] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:07 2022] PUT /identity/v3/projects/cd4daec70cc246f2bde3c88398ad6488/users/fbec55028f1d40e09951cb1a76999f28/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:07.356158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee3a8d-63d3-4b0e-8510-e897517a52e4 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:07.370310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee3a8d-63d3-4b0e-8510-e897517a52e4 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:07.405932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:07.432834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:07.433479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:07.433764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:07.434041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:07.434690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:07.436872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:07.447099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 796/1604] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:07.451344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648940-ce6c-4b86-bb0d-4acc425886bd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:07.458075 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:07.465105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb648940-ce6c-4b86-bb0d-4acc425886bd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:07.470911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eeef70-f243-4f5f-bc8e-a1085ad3dbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:07.471617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeef70-f243-4f5f-bc8e-a1085ad3dbed admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:07.471899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeef70-f243-4f5f-bc8e-a1085ad3dbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:07.472171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eeef70-f243-4f5f-bc8e-a1085ad3dbed admin admin] PATH_INFO: `/v3/projects/cd4daec70cc246f2bde3c88398ad6488/users/fbec55028f1d40e09951cb1a76999f28/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:07.474073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eeef70-f243-4f5f-bc8e-a1085ad3dbed admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd4daec70cc246f2bde3c88398ad6488, user_id=fbec55028f1d40e09951cb1a76999f28, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:07.476879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eeef70-f243-4f5f-bc8e-a1085ad3dbed admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:07.489865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 809/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:07 2022] PUT /identity/v3/projects/cd4daec70cc246f2bde3c88398ad6488/users/fbec55028f1d40e09951cb1a76999f28/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:07.831715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:07.833527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:07.848520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988faa6-8153-4e08-a45c-31b2072874a1 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:07.937624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:07.938345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:07.938645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:07.938932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] PATH_INFO: `/v3/users/729295e0893a4c989e4a1113691f7491` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:07.949812 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=729295e0893a4c989e4a1113691f7491)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:07.958166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:08.038007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '729295e0893a4c989e4a1113691f7491'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:08.054204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '729295e0893a4c989e4a1113691f7491'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:08.054974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '729295e0893a4c989e4a1113691f7491'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:08.066090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '729295e0893a4c989e4a1113691f7491'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:08.075658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '729295e0893a4c989e4a1113691f7491'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:08.083564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 797/1606] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:07 2022] DELETE /identity/v3/users/729295e0893a4c989e4a1113691f7491 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:08.088106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60388-a2cf-4f8a-a148-6a786361e82b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:08.088581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60388-a2cf-4f8a-a148-6a786361e82b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:08.088833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a60388-a2cf-4f8a-a148-6a786361e82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:08.128317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-92a60388-a2cf-4f8a-a148-6a786361e82b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:08.129688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-92a60388-a2cf-4f8a-a148-6a786361e82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:08.219217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 810/1607] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:08.233165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:08.234900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:08.343892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617605a-e418-4f7a-8fc8-233a21a83fbb admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:08.350400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:08.351459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:08.351819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:08.352192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:08.367658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:08.372991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:08.375891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:08.393014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 798/1608] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:08.796151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f425b-ccd5-4b49-a768-9debfa1cee38 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:08.796740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f425b-ccd5-4b49-a768-9debfa1cee38 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:08.796740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f425b-ccd5-4b49-a768-9debfa1cee38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:08.828125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0e4f425b-ccd5-4b49-a768-9debfa1cee38 None None] MFA Rules not processed for user `fbec55028f1d40e09951cb1a76999f28`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:08.829538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0e4f425b-ccd5-4b49-a768-9debfa1cee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:08.908685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:08.910131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 811/1609] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:08.910247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:08.926945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f425b-ccd5-4b49-a768-9debfa1cee38 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:08.930304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8ab51-d0b3-4853-9293-ed49df7cd20c service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:08.935511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:08.936918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0e4f425b-ccd5-4b49-a768-9debfa1cee38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:08.937142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:08.937432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:08.937712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] PATH_INFO: `/v3/projects/13bdc95ac164408b9114a5cb69c9467a` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:08.938892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bdc95ac164408b9114a5cb69c9467a)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:08.940963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:09.014825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f425b-ccd5-4b49-a768-9debfa1cee38 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:09.024788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bdc95ac164408b9114a5cb69c9467a', 'request_id': 'req-578d4cc8-0719-4fe4-b841-1c454468a9b0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.044521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bdc95ac164408b9114a5cb69c9467a', 'request_id': 'req-578d4cc8-0719-4fe4-b841-1c454468a9b0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.056906 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bdc95ac164408b9114a5cb69c9467a', 'request_id': 'req-578d4cc8-0719-4fe4-b841-1c454468a9b0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.075419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] The token cache is being invalidate because project 13bdc95ac164408b9114a5cb69c9467a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:09.075953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.077637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.079267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:09.085664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 799/1610] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:08 2022] DELETE /identity/v3/projects/13bdc95ac164408b9114a5cb69c9467a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:09.089285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:09.092992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:09.156825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:09.157373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:09.157670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:09.157953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:09.160662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:09.202599 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d4cc8-0719-4fe4-b841-1c454468a9b0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:09.210130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:09.210975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:09.211810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:09.212233 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] PATH_INFO: `/v3/users/b71bad03bc6941ce9f7629c98cd1954f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:09.215925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71bad03bc6941ce9f7629c98cd1954f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:09.217488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:09.254672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:09.257261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:09.273102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 812/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:09.286633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71bad03bc6941ce9f7629c98cd1954f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.301007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71bad03bc6941ce9f7629c98cd1954f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.302927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71bad03bc6941ce9f7629c98cd1954f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.329792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71bad03bc6941ce9f7629c98cd1954f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.339724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a859efa9-c6ac-47ce-93f9-3b7c1fa0b8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71bad03bc6941ce9f7629c98cd1954f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.343117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:09.360040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 800/1612] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:09 2022] DELETE /identity/v3/users/b71bad03bc6941ce9f7629c98cd1954f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:09.371121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:09.373280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:09.396206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292d818-5042-46f0-9f45-1386dee0b7fb service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:09.511566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:09.512295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:09.512668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:09.512859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] PATH_INFO: `/v3/users/1c5f2f1df3e7472e8fa90a2bd8acbe56` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:09.514547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5f2f1df3e7472e8fa90a2bd8acbe56)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:09.516962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:09.628804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5f2f1df3e7472e8fa90a2bd8acbe56'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.670214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5f2f1df3e7472e8fa90a2bd8acbe56'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.673349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5f2f1df3e7472e8fa90a2bd8acbe56'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.702158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5f2f1df3e7472e8fa90a2bd8acbe56'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.702985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5f2f1df3e7472e8fa90a2bd8acbe56'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:09.728858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 813/1613] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:09 2022] DELETE /identity/v3/users/1c5f2f1df3e7472e8fa90a2bd8acbe56 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:09.733858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803b9a4-6677-4b8a-bcb4-23d13f869b54 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:09.734244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803b9a4-6677-4b8a-bcb4-23d13f869b54 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:09.734600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5803b9a4-6677-4b8a-bcb4-23d13f869b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:09.762938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5803b9a4-6677-4b8a-bcb4-23d13f869b54 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:09.765871 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5803b9a4-6677-4b8a-bcb4-23d13f869b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:09.890609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 801/1614] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:09.896421 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:09.897772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:09.906108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:09.983952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803b9a4-6677-4b8a-bcb4-23d13f869b54 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:09.985876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5803b9a4-6677-4b8a-bcb4-23d13f869b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:10.019977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5803b9a4-6677-4b8a-bcb4-23d13f869b54 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:10.059328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:10.060004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:10.060262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:10.060544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] PATH_INFO: `/v3/projects/3cff8343a10d4a3c810bf2e587de5bb9` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:10.064464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cff8343a10d4a3c810bf2e587de5bb9)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:10.075555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:10.103808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b1925-6852-45c6-b7f7-3c44953f6a44 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:10.123341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b6566-4baa-4414-95db-43ef70fc9803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:10.124035 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6566-4baa-4414-95db-43ef70fc9803 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:10.124760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6566-4baa-4414-95db-43ef70fc9803 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:10.124760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b6566-4baa-4414-95db-43ef70fc9803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:10.161900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b6566-4baa-4414-95db-43ef70fc9803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:10.175823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b6566-4baa-4414-95db-43ef70fc9803 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:10.219732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 814/1615] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:10.221478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cff8343a10d4a3c810bf2e587de5bb9', 'request_id': 'req-6032aecb-038c-4b77-8a75-fee46b5f4480'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:10.232585 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cff8343a10d4a3c810bf2e587de5bb9', 'request_id': 'req-6032aecb-038c-4b77-8a75-fee46b5f4480'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:10.247437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cff8343a10d4a3c810bf2e587de5bb9', 'request_id': 'req-6032aecb-038c-4b77-8a75-fee46b5f4480'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:10.264103 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] The token cache is being invalidate because project 3cff8343a10d4a3c810bf2e587de5bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:10.264393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:10.265421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6032aecb-038c-4b77-8a75-fee46b5f4480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:10.268588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 802/1616] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:09 2022] DELETE /identity/v3/projects/3cff8343a10d4a3c810bf2e587de5bb9 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:10.273538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fcbf4-e32b-4440-9c14-66b39bc624ac None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:10.273830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fcbf4-e32b-4440-9c14-66b39bc624ac None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:10.274200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fcbf4-e32b-4440-9c14-66b39bc624ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:10.314733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b61fcbf4-e32b-4440-9c14-66b39bc624ac None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:10.316362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b61fcbf4-e32b-4440-9c14-66b39bc624ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:10.438177 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 815/1617] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:10.447900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb2e00-e03b-4ba0-b9ea-91a0d721a60c None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:10.448185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb2e00-e03b-4ba0-b9ea-91a0d721a60c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:10.448523 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb2e00-e03b-4ba0-b9ea-91a0d721a60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:10.480819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-23bb2e00-e03b-4ba0-b9ea-91a0d721a60c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:10.482050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-23bb2e00-e03b-4ba0-b9ea-91a0d721a60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:10.509467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 803/1618] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:10.515026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fcbf4-e32b-4440-9c14-66b39bc624ac None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:10.528037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fcbf4-e32b-4440-9c14-66b39bc624ac None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:10.538709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:10.540851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:10.552301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:10.554913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:10.555992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:10.556607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:10.558411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:10.594963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 816/1619] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:10.604173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb2e00-e03b-4ba0-b9ea-91a0d721a60c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:10.628653 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb2e00-e03b-4ba0-b9ea-91a0d721a60c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:10.640385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7fc4d-c812-4e58-a250-867d733593fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:10.641538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7fc4d-c812-4e58-a250-867d733593fe admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:10.642060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7fc4d-c812-4e58-a250-867d733593fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:10.642601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7fc4d-c812-4e58-a250-867d733593fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:10.646216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7fc4d-c812-4e58-a250-867d733593fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:10.646216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7fc4d-c812-4e58-a250-867d733593fe admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:10.725338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 804/1620] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:10 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:10.728660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:10.759391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fd157-c464-4d09-a337-fc51db59c8bc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:10.869319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:10.870253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:10.870682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:10.871143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:10.872041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:10.875275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:10.924313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 817/1621] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:10 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:10.926772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7fc4d-c812-4e58-a250-867d733593fe admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:10.954204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:10.957713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:10.964908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7fc4d-c812-4e58-a250-867d733593fe admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:10.974634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:10.975401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:10.976080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:10.976857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:10.978768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:10.987257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.021596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 805/1622] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:11.025714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.032584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:11.039522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:11.080605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963bee8-d1e1-451d-910f-1128edd84896 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.088844 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.090220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.091356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.091714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] PATH_INFO: `/v3/projects/59dd0055a7684dc2a7e0c94540d10f48` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.093784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59dd0055a7684dc2a7e0c94540d10f48)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.096229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.143568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5940fc6-5f6c-4137-919f-fa0c6cf94d6a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.149637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cfde3-f8f7-4718-86cc-61202bf9fead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.151089 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cfde3-f8f7-4718-86cc-61202bf9fead service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.151665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cfde3-f8f7-4718-86cc-61202bf9fead service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.152155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cfde3-f8f7-4718-86cc-61202bf9fead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.154489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d14cfde3-f8f7-4718-86cc-61202bf9fead service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:11.175252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:11.211998 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59dd0055a7684dc2a7e0c94540d10f48', 'request_id': 'req-e3d9f44a-0e39-4013-ad2d-d40aac313321'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:11.222841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59dd0055a7684dc2a7e0c94540d10f48', 'request_id': 'req-e3d9f44a-0e39-4013-ad2d-d40aac313321'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:11.237032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59dd0055a7684dc2a7e0c94540d10f48', 'request_id': 'req-e3d9f44a-0e39-4013-ad2d-d40aac313321'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:11.261220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] The token cache is being invalidate because project 59dd0055a7684dc2a7e0c94540d10f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:11.261692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:11.262441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:11.264519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 806/1623] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:11 2022] DELETE /identity/v3/projects/59dd0055a7684dc2a7e0c94540d10f48 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:11.269552 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.270888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cfde3-f8f7-4718-86cc-61202bf9fead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.271386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:11.272389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14cfde3-f8f7-4718-86cc-61202bf9fead service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.287274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 818/1624] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:11.290443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ade8f-318c-4222-a6e3-96b6d9305f60 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.291213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ade8f-318c-4222-a6e3-96b6d9305f60 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.291502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ade8f-318c-4222-a6e3-96b6d9305f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.313874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f40ade8f-318c-4222-a6e3-96b6d9305f60 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:11.315126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f40ade8f-318c-4222-a6e3-96b6d9305f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:11.366943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9f44a-0e39-4013-ad2d-d40aac313321 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.374191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03263e-b5fa-4643-a97e-9bd95deb8e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.375235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03263e-b5fa-4643-a97e-9bd95deb8e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.375519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03263e-b5fa-4643-a97e-9bd95deb8e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.375797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03263e-b5fa-4643-a97e-9bd95deb8e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.376666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03263e-b5fa-4643-a97e-9bd95deb8e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.378409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03263e-b5fa-4643-a97e-9bd95deb8e49 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.399499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 807/1625] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:11.399680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 819/1626] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:11.403659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ade8f-318c-4222-a6e3-96b6d9305f60 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.404498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78e51-ea6c-4ab7-ab64-78e1da7fa733 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.405074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78e51-ea6c-4ab7-ab64-78e1da7fa733 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.405661 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be78e51-ea6c-4ab7-ab64-78e1da7fa733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.416886 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ade8f-318c-4222-a6e3-96b6d9305f60 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.421179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c31d1-123d-4fbb-83cd-27a9f9226146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.422030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c31d1-123d-4fbb-83cd-27a9f9226146 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.422310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c31d1-123d-4fbb-83cd-27a9f9226146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.422586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c31d1-123d-4fbb-83cd-27a9f9226146 admin admin] PATH_INFO: `/v3/projects/322d5ac0e3054014a08fd8135e684833/users/90c7b7b9798c44d28d370b0a97e19831/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.430946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9be78e51-ea6c-4ab7-ab64-78e1da7fa733 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:11.432469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9be78e51-ea6c-4ab7-ab64-78e1da7fa733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:11.444009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c31d1-123d-4fbb-83cd-27a9f9226146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322d5ac0e3054014a08fd8135e684833, user_id=90c7b7b9798c44d28d370b0a97e19831, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.446965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c31d1-123d-4fbb-83cd-27a9f9226146 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.447759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 808/1627] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:11.448800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be78e51-ea6c-4ab7-ab64-78e1da7fa733 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.461516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be78e51-ea6c-4ab7-ab64-78e1da7fa733 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.468072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 820/1628] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:11 2022] PUT /identity/v3/projects/322d5ac0e3054014a08fd8135e684833/users/90c7b7b9798c44d28d370b0a97e19831/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:11.479604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7951-410f-4daf-b5ef-538f8f90eb5b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.480538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7951-410f-4daf-b5ef-538f8f90eb5b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.481124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6d7951-410f-4daf-b5ef-538f8f90eb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.521530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3c6d7951-410f-4daf-b5ef-538f8f90eb5b None None] MFA Rules not processed for user `90c7b7b9798c44d28d370b0a97e19831`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:11.522777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3c6d7951-410f-4daf-b5ef-538f8f90eb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:11.549431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.550401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.550822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.551268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.551994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:11.552309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.554013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.567431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 809/1629] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:11.571451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.583787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdfc89-45b3-4294-8386-6d061e61febe admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.588619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91712e7b-2ba5-4fee-8962-6ed8bfaf9885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.589236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91712e7b-2ba5-4fee-8962-6ed8bfaf9885 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.589528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91712e7b-2ba5-4fee-8962-6ed8bfaf9885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.589807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91712e7b-2ba5-4fee-8962-6ed8bfaf9885 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.590579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91712e7b-2ba5-4fee-8962-6ed8bfaf9885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.592997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:11.593247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91712e7b-2ba5-4fee-8962-6ed8bfaf9885 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.627318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 821/1630] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:11.629809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 810/1631] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:11 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:11.633954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d7951-410f-4daf-b5ef-538f8f90eb5b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.634664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91712e7b-2ba5-4fee-8962-6ed8bfaf9885 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.645465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6d7951-410f-4daf-b5ef-538f8f90eb5b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.666960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:11.704294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91712e7b-2ba5-4fee-8962-6ed8bfaf9885 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.704738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb75ad0-3671-4d0e-bd85-9e869ac8c8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.705395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75ad0-3671-4d0e-bd85-9e869ac8c8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.705726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75ad0-3671-4d0e-bd85-9e869ac8c8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.706055 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb75ad0-3671-4d0e-bd85-9e869ac8c8ee admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.706824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb75ad0-3671-4d0e-bd85-9e869ac8c8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.709195 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb75ad0-3671-4d0e-bd85-9e869ac8c8ee admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.709435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815b654-34f5-4573-b37c-fc9665ca1a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.710035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815b654-34f5-4573-b37c-fc9665ca1a3f admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.710331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815b654-34f5-4573-b37c-fc9665ca1a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.710610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815b654-34f5-4573-b37c-fc9665ca1a3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.711267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815b654-34f5-4573-b37c-fc9665ca1a3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.713193 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2815b654-34f5-4573-b37c-fc9665ca1a3f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.731935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 822/1632] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:11 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:11.736222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb75ad0-3671-4d0e-bd85-9e869ac8c8ee admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.745674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 811/1633] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:11 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:11.745869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb75ad0-3671-4d0e-bd85-9e869ac8c8ee admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.749661 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815b654-34f5-4573-b37c-fc9665ca1a3f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.823289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b844bb0-a222-4c2e-a37b-1524f61fc96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.824043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2815b654-34f5-4573-b37c-fc9665ca1a3f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.824152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b844bb0-a222-4c2e-a37b-1524f61fc96b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.824514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b844bb0-a222-4c2e-a37b-1524f61fc96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.824862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b844bb0-a222-4c2e-a37b-1524f61fc96b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.825600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b844bb0-a222-4c2e-a37b-1524f61fc96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.828044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b844bb0-a222-4c2e-a37b-1524f61fc96b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.829794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16c1da-d944-41e0-930f-9d98bb8dffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.830364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16c1da-d944-41e0-930f-9d98bb8dffdc admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.830996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16c1da-d944-41e0-930f-9d98bb8dffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.830996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16c1da-d944-41e0-930f-9d98bb8dffdc admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.831598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16c1da-d944-41e0-930f-9d98bb8dffdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.833935 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa16c1da-d944-41e0-930f-9d98bb8dffdc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.839702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 823/1634] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:11.844280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b844bb0-a222-4c2e-a37b-1524f61fc96b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.855029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b844bb0-a222-4c2e-a37b-1524f61fc96b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.860202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb433816-ecb3-4cb0-93d0-0ab8dd974ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.860500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 812/1635] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:11 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:11.861310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433816-ecb3-4cb0-93d0-0ab8dd974ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.861310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433816-ecb3-4cb0-93d0-0ab8dd974ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.861492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16c1da-d944-41e0-930f-9d98bb8dffdc admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.861576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433816-ecb3-4cb0-93d0-0ab8dd974ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.862259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb433816-ecb3-4cb0-93d0-0ab8dd974ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.863973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb433816-ecb3-4cb0-93d0-0ab8dd974ea1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.875384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16c1da-d944-41e0-930f-9d98bb8dffdc admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.876255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 824/1636] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:11.880372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb433816-ecb3-4cb0-93d0-0ab8dd974ea1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.882333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c549-e186-46cf-8516-3887d5c5c4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.883015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c549-e186-46cf-8516-3887d5c5c4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.883383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c549-e186-46cf-8516-3887d5c5c4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.883764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8c549-e186-46cf-8516-3887d5c5c4f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.884539 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8c549-e186-46cf-8516-3887d5c5c4f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.886631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f8c549-e186-46cf-8516-3887d5c5c4f2 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.893007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb433816-ecb3-4cb0-93d0-0ab8dd974ea1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.901007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3e838-706f-4f43-856b-33387fb5ba52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.901540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 813/1637] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:11.902267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e838-706f-4f43-856b-33387fb5ba52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.902743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e838-706f-4f43-856b-33387fb5ba52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.903071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3e838-706f-4f43-856b-33387fb5ba52 admin admin] PATH_INFO: `/v3/projects/3d7644089a934ed9b9ce5e07345da0c8/users/34addecdb9a54cd29874043ea08e4d28/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.906631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c549-e186-46cf-8516-3887d5c5c4f2 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:11.922642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3e838-706f-4f43-856b-33387fb5ba52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d7644089a934ed9b9ce5e07345da0c8, user_id=34addecdb9a54cd29874043ea08e4d28, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.924073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f8c549-e186-46cf-8516-3887d5c5c4f2 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:11.925436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3e838-706f-4f43-856b-33387fb5ba52 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:11.931618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884645-149f-4c47-8c47-2eff999dfad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:11.932486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884645-149f-4c47-8c47-2eff999dfad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.932781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884645-149f-4c47-8c47-2eff999dfad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.933215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884645-149f-4c47-8c47-2eff999dfad9 admin admin] PATH_INFO: `/v3/projects/cde2fc485e45491b8298690cec792053/users/15775502e6774d0194b798ae65b487f5/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.949403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 825/1638] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:11 2022] PUT /identity/v3/projects/3d7644089a934ed9b9ce5e07345da0c8/users/34addecdb9a54cd29874043ea08e4d28/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:11.961816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080393dc-6b2b-4d2c-a054-7dc9b1fbd07e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:11.962027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080393dc-6b2b-4d2c-a054-7dc9b1fbd07e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:11.962304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080393dc-6b2b-4d2c-a054-7dc9b1fbd07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:11.969087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24884645-149f-4c47-8c47-2eff999dfad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde2fc485e45491b8298690cec792053, user_id=15775502e6774d0194b798ae65b487f5, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:11.974642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24884645-149f-4c47-8c47-2eff999dfad9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:12.015763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 814/1639] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:11 2022] PUT /identity/v3/projects/cde2fc485e45491b8298690cec792053/users/15775502e6774d0194b798ae65b487f5/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:12.018056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-080393dc-6b2b-4d2c-a054-7dc9b1fbd07e None None] MFA Rules not processed for user `34addecdb9a54cd29874043ea08e4d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:12.019150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-080393dc-6b2b-4d2c-a054-7dc9b1fbd07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:12.019289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884645-149f-4c47-8c47-2eff999dfad9 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:12.039243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:12.112995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 826/1640] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:12.126006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884645-149f-4c47-8c47-2eff999dfad9 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:12.127274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080393dc-6b2b-4d2c-a054-7dc9b1fbd07e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:12.128736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-080393dc-6b2b-4d2c-a054-7dc9b1fbd07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:12.131157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708d61a-dda5-441e-afbe-910c82a695fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:12.131848 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708d61a-dda5-441e-afbe-910c82a695fc admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:12.132133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708d61a-dda5-441e-afbe-910c82a695fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:12.132407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708d61a-dda5-441e-afbe-910c82a695fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:12.133051 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708d61a-dda5-441e-afbe-910c82a695fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:12.135282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708d61a-dda5-441e-afbe-910c82a695fc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:12.147961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 815/1641] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:12.152115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708d61a-dda5-441e-afbe-910c82a695fc admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:12.158532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:12.166266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708d61a-dda5-441e-afbe-910c82a695fc admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:12.172612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be35d30-52f8-492d-b996-48d2b037c60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:12.173418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35d30-52f8-492d-b996-48d2b037c60d admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:12.173888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35d30-52f8-492d-b996-48d2b037c60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:12.174268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be35d30-52f8-492d-b996-48d2b037c60d admin admin] PATH_INFO: `/v3/projects/cde2fc485e45491b8298690cec792053/users/15775502e6774d0194b798ae65b487f5/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:12.176656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35d30-52f8-492d-b996-48d2b037c60d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde2fc485e45491b8298690cec792053, user_id=15775502e6774d0194b798ae65b487f5, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:12.180713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be35d30-52f8-492d-b996-48d2b037c60d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:12.194329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 816/1642] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:12 2022] PUT /identity/v3/projects/cde2fc485e45491b8298690cec792053/users/15775502e6774d0194b798ae65b487f5/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:12.208222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72537ff0-2ad4-45c5-bc0f-790cc27622e7 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:12.208809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72537ff0-2ad4-45c5-bc0f-790cc27622e7 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:12.209867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72537ff0-2ad4-45c5-bc0f-790cc27622e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:12.209973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080393dc-6b2b-4d2c-a054-7dc9b1fbd07e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:12.247914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-72537ff0-2ad4-45c5-bc0f-790cc27622e7 None None] MFA Rules not processed for user `15775502e6774d0194b798ae65b487f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:12.250239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-72537ff0-2ad4-45c5-bc0f-790cc27622e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:12.279962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dff4a7-171e-4b4b-ab4c-e5eceee6e4f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:12.280847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dff4a7-171e-4b4b-ab4c-e5eceee6e4f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:12.281378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dff4a7-171e-4b4b-ab4c-e5eceee6e4f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:12.281662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dff4a7-171e-4b4b-ab4c-e5eceee6e4f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:12.363202 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 817/1643] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:12.371848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dff4a7-171e-4b4b-ab4c-e5eceee6e4f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:12.374688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dff4a7-171e-4b4b-ab4c-e5eceee6e4f7 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:12.377370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72537ff0-2ad4-45c5-bc0f-790cc27622e7 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:12.379331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-72537ff0-2ad4-45c5-bc0f-790cc27622e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:12.395199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72537ff0-2ad4-45c5-bc0f-790cc27622e7 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:12.399152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 827/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:12.407648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed3d31-07b8-49f4-a1ac-d66e5cb0af54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:12.408515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed3d31-07b8-49f4-a1ac-d66e5cb0af54 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:12.408861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed3d31-07b8-49f4-a1ac-d66e5cb0af54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:12.409315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed3d31-07b8-49f4-a1ac-d66e5cb0af54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:12.421921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed3d31-07b8-49f4-a1ac-d66e5cb0af54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:12.423677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed3d31-07b8-49f4-a1ac-d66e5cb0af54 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:12.434388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 818/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:12.548879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dff4a7-171e-4b4b-ab4c-e5eceee6e4f7 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:12.559267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dff4a7-171e-4b4b-ab4c-e5eceee6e4f7 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:12.564574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6787b4-cbe9-4532-98b2-9bb41ee9b61b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:12.565510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6787b4-cbe9-4532-98b2-9bb41ee9b61b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:12.565510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6787b4-cbe9-4532-98b2-9bb41ee9b61b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:12.565766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6787b4-cbe9-4532-98b2-9bb41ee9b61b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:12.600518 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:12.632012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6787b4-cbe9-4532-98b2-9bb41ee9b61b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:12.633398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6787b4-cbe9-4532-98b2-9bb41ee9b61b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:12.643376 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 828/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:15.456111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed3d31-07b8-49f4-a1ac-d66e5cb0af54 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:15.538830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed3d31-07b8-49f4-a1ac-d66e5cb0af54 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:15.544139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:15.544780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:15.545122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:15.545428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] PATH_INFO: `/v3/users/90c7b7b9798c44d28d370b0a97e19831` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:15.547040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c7b7b9798c44d28d370b0a97e19831)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:15.549379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:15.672149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c7b7b9798c44d28d370b0a97e19831'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:15.689672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:15.691960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c7b7b9798c44d28d370b0a97e19831'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:15.694080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c7b7b9798c44d28d370b0a97e19831'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:15.706544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c7b7b9798c44d28d370b0a97e19831'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:15.715767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c7b7b9798c44d28d370b0a97e19831'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:15.726089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 819/1647] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:15 2022] DELETE /identity/v3/users/90c7b7b9798c44d28d370b0a97e19831 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:15.727705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954850-c9c0-4a4e-a127-2215c307fd86 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:15.728149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954850-c9c0-4a4e-a127-2215c307fd86 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:15.728312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10954850-c9c0-4a4e-a127-2215c307fd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:15.773763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-10954850-c9c0-4a4e-a127-2215c307fd86 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:15.777802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-10954850-c9c0-4a4e-a127-2215c307fd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:15.929514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 829/1648] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:15.950071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:15.953465 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:16.147763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec524a-2f29-4bdf-be0d-c99c30a8e5f8 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:16.165524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a4f69-2940-4d4a-b12f-8c9e7f0cc498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:16.166165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a4f69-2940-4d4a-b12f-8c9e7f0cc498 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:16.166451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a4f69-2940-4d4a-b12f-8c9e7f0cc498 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:16.166724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a4f69-2940-4d4a-b12f-8c9e7f0cc498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:16.187371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a4f69-2940-4d4a-b12f-8c9e7f0cc498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:16.188425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a4f69-2940-4d4a-b12f-8c9e7f0cc498 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:16.215288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 820/1649] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:16.677110 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10954850-c9c0-4a4e-a127-2215c307fd86 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:16.678685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-10954850-c9c0-4a4e-a127-2215c307fd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:16.696313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10954850-c9c0-4a4e-a127-2215c307fd86 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:16.701260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:16.701962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:16.702295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:16.702576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] PATH_INFO: `/v3/projects/322d5ac0e3054014a08fd8135e684833` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:16.703751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] RBAC: Authorizing `identity:delete_project(project_id=322d5ac0e3054014a08fd8135e684833)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:16.707070 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:16.772051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322d5ac0e3054014a08fd8135e684833', 'request_id': 'req-e89f26f5-8aa6-47d2-81ed-66a50536fbda'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:16.778989 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322d5ac0e3054014a08fd8135e684833', 'request_id': 'req-e89f26f5-8aa6-47d2-81ed-66a50536fbda'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:16.788960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322d5ac0e3054014a08fd8135e684833', 'request_id': 'req-e89f26f5-8aa6-47d2-81ed-66a50536fbda'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:16.815672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] The token cache is being invalidate because project 322d5ac0e3054014a08fd8135e684833 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:16.815970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:16.816630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:16.818486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 830/1650] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:16 2022] DELETE /identity/v3/projects/322d5ac0e3054014a08fd8135e684833 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:16.822294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a4f69-2940-4d4a-b12f-8c9e7f0cc498 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:16.834069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a4f69-2940-4d4a-b12f-8c9e7f0cc498 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:16.902670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:16.903728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:16.904012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:16.904326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] PATH_INFO: `/v3/users/15775502e6774d0194b798ae65b487f5` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:16.905907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15775502e6774d0194b798ae65b487f5)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:16.908127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:16.978874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15775502e6774d0194b798ae65b487f5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:16.993487 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15775502e6774d0194b798ae65b487f5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:16.994245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15775502e6774d0194b798ae65b487f5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:17.003717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15775502e6774d0194b798ae65b487f5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:17.021799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-efcce607-2cc7-4847-828b-9dbeac45d3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15775502e6774d0194b798ae65b487f5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:17.031255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 821/1651] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:16 2022] DELETE /identity/v3/users/15775502e6774d0194b798ae65b487f5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:17.646186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:17.647871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:17.672735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f26f5-8aa6-47d2-81ed-66a50536fbda admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:17.868312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:17.893763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:17.896951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:17.897294 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:17.899161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] PATH_INFO: `/v3/projects/cde2fc485e45491b8298690cec792053` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:17.901340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde2fc485e45491b8298690cec792053)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:17.903356 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:17.985198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde2fc485e45491b8298690cec792053', 'request_id': 'req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:17.992713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde2fc485e45491b8298690cec792053', 'request_id': 'req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:18.004093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde2fc485e45491b8298690cec792053', 'request_id': 'req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:18.024263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] The token cache is being invalidate because project cde2fc485e45491b8298690cec792053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:18.024784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:18.025679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35312c6a-0c32-4ca5-acd3-a9d3d5315d60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:18.028432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 831/1652] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:17 2022] DELETE /identity/v3/projects/cde2fc485e45491b8298690cec792053 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:18.034105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ddba-632c-4773-a98e-9123084882f8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.035228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ddba-632c-4773-a98e-9123084882f8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.035696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891ddba-632c-4773-a98e-9123084882f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.047164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:18.067351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6891ddba-632c-4773-a98e-9123084882f8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:18.074070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6891ddba-632c-4773-a98e-9123084882f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:18.225329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c878b-199a-413e-867f-c5dbbe1cbbd8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.225618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c878b-199a-413e-867f-c5dbbe1cbbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.225897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c878b-199a-413e-867f-c5dbbe1cbbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.236678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 822/1653] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:18.274741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b33c878b-199a-413e-867f-c5dbbe1cbbd8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:18.276611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b33c878b-199a-413e-867f-c5dbbe1cbbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:18.285182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 832/1654] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:18.289967 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ddba-632c-4773-a98e-9123084882f8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:18.302977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891ddba-632c-4773-a98e-9123084882f8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:18.320487 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:18.321736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.322214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.322676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.323537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:18.324052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:18.325999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:18.345385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c878b-199a-413e-867f-c5dbbe1cbbd8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:18.350784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 823/1655] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:18.357149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c878b-199a-413e-867f-c5dbbe1cbbd8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:18.364111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83ad89-e5c1-49ad-80a1-63268eee6f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:18.364969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83ad89-e5c1-49ad-80a1-63268eee6f18 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.365445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83ad89-e5c1-49ad-80a1-63268eee6f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.365924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f83ad89-e5c1-49ad-80a1-63268eee6f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.366750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83ad89-e5c1-49ad-80a1-63268eee6f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:18.369168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f83ad89-e5c1-49ad-80a1-63268eee6f18 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:18.419477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 833/1656] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:18 2022] POST /identity/v3/projects => generated 389 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:18.425606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:18.455411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67570df7-6cd2-40ef-9aec-9366ff17ac0c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:18.545845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888deda-9a07-450b-bc29-a1745530ac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:18.546519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888deda-9a07-450b-bc29-a1745530ac3c admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.546829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888deda-9a07-450b-bc29-a1745530ac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.547149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888deda-9a07-450b-bc29-a1745530ac3c admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.547817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888deda-9a07-450b-bc29-a1745530ac3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:18.549758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888deda-9a07-450b-bc29-a1745530ac3c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:18.586737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 824/1657] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:18 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:18.588206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83ad89-e5c1-49ad-80a1-63268eee6f18 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:18.601037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f83ad89-e5c1-49ad-80a1-63268eee6f18 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:18.606820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe19c2-712d-42e2-b47f-fa6eb34235f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:18.608085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe19c2-712d-42e2-b47f-fa6eb34235f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.608565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe19c2-712d-42e2-b47f-fa6eb34235f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.609021 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe19c2-712d-42e2-b47f-fa6eb34235f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.609899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe19c2-712d-42e2-b47f-fa6eb34235f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:18.624000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe19c2-712d-42e2-b47f-fa6eb34235f7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:18.637186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 834/1658] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:18.641756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888deda-9a07-450b-bc29-a1745530ac3c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:18.655104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888deda-9a07-450b-bc29-a1745530ac3c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:18.658428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f154e-e33e-440d-a8e1-2d6a6a4b272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:18.659159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f154e-e33e-440d-a8e1-2d6a6a4b272e admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.659612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f154e-e33e-440d-a8e1-2d6a6a4b272e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.659958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f154e-e33e-440d-a8e1-2d6a6a4b272e admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.660677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f154e-e33e-440d-a8e1-2d6a6a4b272e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:18.662339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f154e-e33e-440d-a8e1-2d6a6a4b272e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:18.674849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 825/1659] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:18.679097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe19c2-712d-42e2-b47f-fa6eb34235f7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:18.691504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe19c2-712d-42e2-b47f-fa6eb34235f7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:18.697164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeff096-73e9-41f4-b50b-1fed92afe1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:18.697945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff096-73e9-41f4-b50b-1fed92afe1fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.698264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff096-73e9-41f4-b50b-1fed92afe1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.698608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeff096-73e9-41f4-b50b-1fed92afe1fe admin admin] PATH_INFO: `/v3/projects/d91f7e646cae4e7e883905d33822eb9a/users/f86b1dd0d4434c61999e1c7816fec2da/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.739126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeff096-73e9-41f4-b50b-1fed92afe1fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91f7e646cae4e7e883905d33822eb9a, user_id=f86b1dd0d4434c61999e1c7816fec2da, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:18.742190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeff096-73e9-41f4-b50b-1fed92afe1fe admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:18.762679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 835/1660] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:18 2022] PUT /identity/v3/projects/d91f7e646cae4e7e883905d33822eb9a/users/f86b1dd0d4434c61999e1c7816fec2da/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:18.774352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b2a4c-6cf2-402a-8de2-177d411bbe0a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.774642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b2a4c-6cf2-402a-8de2-177d411bbe0a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.774916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b2a4c-6cf2-402a-8de2-177d411bbe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.803618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f51b2a4c-6cf2-402a-8de2-177d411bbe0a None None] MFA Rules not processed for user `f86b1dd0d4434c61999e1c7816fec2da`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:18.804878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f51b2a4c-6cf2-402a-8de2-177d411bbe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:18.872698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:18.879547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 826/1661] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:18.884012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeff096-73e9-41f4-b50b-1fed92afe1fe admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:18.968993 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeff096-73e9-41f4-b50b-1fed92afe1fe admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:18.975217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1634116-71b1-4fd0-8f70-d2109035ce17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:18.975916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1634116-71b1-4fd0-8f70-d2109035ce17 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:18.976275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1634116-71b1-4fd0-8f70-d2109035ce17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:18.976626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1634116-71b1-4fd0-8f70-d2109035ce17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:18.977371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1634116-71b1-4fd0-8f70-d2109035ce17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:18.979393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1634116-71b1-4fd0-8f70-d2109035ce17 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:19.008462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 836/1662] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:18 2022] POST /identity/v3/projects => generated 389 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:19.012768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b2a4c-6cf2-402a-8de2-177d411bbe0a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:19.025206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b2a4c-6cf2-402a-8de2-177d411bbe0a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:19.094307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6eb62-66a4-43ab-bc05-9b82c2c55694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:19.095031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6eb62-66a4-43ab-bc05-9b82c2c55694 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:19.095411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6eb62-66a4-43ab-bc05-9b82c2c55694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:19.095763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6eb62-66a4-43ab-bc05-9b82c2c55694 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:19.096468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6eb62-66a4-43ab-bc05-9b82c2c55694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:19.098843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6eb62-66a4-43ab-bc05-9b82c2c55694 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:19.121311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 827/1663] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:19 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:19.126192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1634116-71b1-4fd0-8f70-d2109035ce17 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:19.133973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:19.138552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1634116-71b1-4fd0-8f70-d2109035ce17 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:19.145413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44e4b0-e05d-49a0-bd1b-eb0f8f7f4df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:19.146290 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44e4b0-e05d-49a0-bd1b-eb0f8f7f4df7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:19.146780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44e4b0-e05d-49a0-bd1b-eb0f8f7f4df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:19.147384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44e4b0-e05d-49a0-bd1b-eb0f8f7f4df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:19.148109 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44e4b0-e05d-49a0-bd1b-eb0f8f7f4df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:19.150119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44e4b0-e05d-49a0-bd1b-eb0f8f7f4df7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:19.165452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 837/1664] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:19.166896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6eb62-66a4-43ab-bc05-9b82c2c55694 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:19.178455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6eb62-66a4-43ab-bc05-9b82c2c55694 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:19.183313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3088f-3f08-4bde-98dc-5d5347aff4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:19.184005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3088f-3f08-4bde-98dc-5d5347aff4df admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:19.184287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3088f-3f08-4bde-98dc-5d5347aff4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:19.184563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3088f-3f08-4bde-98dc-5d5347aff4df admin admin] PATH_INFO: `/v3/projects/25c5c3f7e00949f3b1fb6e8ad5478123/users/571d448418c940a18f591f91f595de03/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:19.202422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3088f-3f08-4bde-98dc-5d5347aff4df admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c5c3f7e00949f3b1fb6e8ad5478123, user_id=571d448418c940a18f591f91f595de03, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:19.205219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3088f-3f08-4bde-98dc-5d5347aff4df admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:19.219598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 828/1665] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:19 2022] PUT /identity/v3/projects/25c5c3f7e00949f3b1fb6e8ad5478123/users/571d448418c940a18f591f91f595de03/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:19.223768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44e4b0-e05d-49a0-bd1b-eb0f8f7f4df7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:19.234982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44e4b0-e05d-49a0-bd1b-eb0f8f7f4df7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:19.294376 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fd3ad-82c2-4cf9-b332-281c61cf8f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:19.295931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd3ad-82c2-4cf9-b332-281c61cf8f43 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:19.296221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd3ad-82c2-4cf9-b332-281c61cf8f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:19.296498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fd3ad-82c2-4cf9-b332-281c61cf8f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:19.297166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fd3ad-82c2-4cf9-b332-281c61cf8f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:19.298775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fd3ad-82c2-4cf9-b332-281c61cf8f43 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:19.312489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 838/1666] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:19.313584 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3088f-3f08-4bde-98dc-5d5347aff4df admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:19.323776 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3088f-3f08-4bde-98dc-5d5347aff4df admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:19.328927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:19.329642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:19.329918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:19.330196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] PATH_INFO: `/v3/projects/25c5c3f7e00949f3b1fb6e8ad5478123/users/571d448418c940a18f591f91f595de03/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:19.332144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c5c3f7e00949f3b1fb6e8ad5478123, user_id=571d448418c940a18f591f91f595de03, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:19.334787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:19.347559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 829/1667] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:19 2022] PUT /identity/v3/projects/25c5c3f7e00949f3b1fb6e8ad5478123/users/571d448418c940a18f591f91f595de03/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:19.359448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345ef7a-5deb-46bd-a8ac-ed62755ae726 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:19.359812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345ef7a-5deb-46bd-a8ac-ed62755ae726 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:19.360182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8345ef7a-5deb-46bd-a8ac-ed62755ae726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:19.390136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8345ef7a-5deb-46bd-a8ac-ed62755ae726 None None] MFA Rules not processed for user `571d448418c940a18f591f91f595de03`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:19.391440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8345ef7a-5deb-46bd-a8ac-ed62755ae726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:19.464996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 839/1668] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:19 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:19.473468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:19.476233 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:19.539028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e4b57-2bce-432c-9cb2-fe860540716b admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:19.542765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d124437-00fc-4438-a1bc-955534c9515f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:19.543483 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124437-00fc-4438-a1bc-955534c9515f service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:19.543836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124437-00fc-4438-a1bc-955534c9515f service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:19.544179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d124437-00fc-4438-a1bc-955534c9515f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:19.553459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d124437-00fc-4438-a1bc-955534c9515f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:19.554897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d124437-00fc-4438-a1bc-955534c9515f service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:19.569608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 830/1669] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:19 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:19.962859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345ef7a-5deb-46bd-a8ac-ed62755ae726 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:20.039357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8345ef7a-5deb-46bd-a8ac-ed62755ae726 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:20.044402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:20.045597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:20.045879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:20.046156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] PATH_INFO: `/v3/users/f86b1dd0d4434c61999e1c7816fec2da` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:20.047397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f86b1dd0d4434c61999e1c7816fec2da)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:20.049670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:20.076657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:20.133321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f86b1dd0d4434c61999e1c7816fec2da'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:20.147851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f86b1dd0d4434c61999e1c7816fec2da'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:20.148786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f86b1dd0d4434c61999e1c7816fec2da'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:20.163935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f86b1dd0d4434c61999e1c7816fec2da'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:20.165251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f86b1dd0d4434c61999e1c7816fec2da'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:20.181399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 840/1670] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:19 2022] DELETE /identity/v3/users/f86b1dd0d4434c61999e1c7816fec2da => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:20.185721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7b133-b3a8-4adf-8c0c-4daf319e2778 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:20.186185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7b133-b3a8-4adf-8c0c-4daf319e2778 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:20.186381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7b133-b3a8-4adf-8c0c-4daf319e2778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:20.208173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-25a7b133-b3a8-4adf-8c0c-4daf319e2778 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:20.209669 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-25a7b133-b3a8-4adf-8c0c-4daf319e2778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:20.268199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:20.282477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 831/1671] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:20.292508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:20.293908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:20.375337 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18c8a4-ed54-4aac-807c-637a3e87de6d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:20.379836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc015a6-4ada-484d-9649-b45b5d883135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:20.380478 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc015a6-4ada-484d-9649-b45b5d883135 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:20.380764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc015a6-4ada-484d-9649-b45b5d883135 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:20.381042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc015a6-4ada-484d-9649-b45b5d883135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:20.393203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc015a6-4ada-484d-9649-b45b5d883135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:20.394960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc015a6-4ada-484d-9649-b45b5d883135 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:20.416325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 841/1672] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:20.899467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7b133-b3a8-4adf-8c0c-4daf319e2778 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:20.901285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-25a7b133-b3a8-4adf-8c0c-4daf319e2778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:20.916458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7b133-b3a8-4adf-8c0c-4daf319e2778 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:20.930657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:20.931574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:20.931865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:20.932145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] PATH_INFO: `/v3/projects/d91f7e646cae4e7e883905d33822eb9a` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:20.933318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91f7e646cae4e7e883905d33822eb9a)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:20.935286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:21.005793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91f7e646cae4e7e883905d33822eb9a', 'request_id': 'req-6691e452-4d30-411f-92e9-599a61f8a3cb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.031875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91f7e646cae4e7e883905d33822eb9a', 'request_id': 'req-6691e452-4d30-411f-92e9-599a61f8a3cb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.042615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91f7e646cae4e7e883905d33822eb9a', 'request_id': 'req-6691e452-4d30-411f-92e9-599a61f8a3cb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.061239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] The token cache is being invalidate because project d91f7e646cae4e7e883905d33822eb9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:21.061603 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.063135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.067891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 832/1673] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:20 2022] DELETE /identity/v3/projects/d91f7e646cae4e7e883905d33822eb9a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:21.072096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc015a6-4ada-484d-9649-b45b5d883135 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:21.083992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc015a6-4ada-484d-9649-b45b5d883135 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:21.163653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:21.164299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:21.164591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:21.164987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] PATH_INFO: `/v3/users/571d448418c940a18f591f91f595de03` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:21.166142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571d448418c940a18f591f91f595de03)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:21.168171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:21.219402 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:21.226754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571d448418c940a18f591f91f595de03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.249012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571d448418c940a18f591f91f595de03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.249903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571d448418c940a18f591f91f595de03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.258142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571d448418c940a18f591f91f595de03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.259185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29ad269d-f57f-49f8-a3a2-768bc04d5788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571d448418c940a18f591f91f595de03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.275428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 842/1674] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:21 2022] DELETE /identity/v3/users/571d448418c940a18f591f91f595de03 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:21.726084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:21.727649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:21.747196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6691e452-4d30-411f-92e9-599a61f8a3cb admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:21.837030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:21.837860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:21.838210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:21.838562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] PATH_INFO: `/v3/projects/25c5c3f7e00949f3b1fb6e8ad5478123` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:21.840177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c5c3f7e00949f3b1fb6e8ad5478123)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:21.843096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:21.909394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c5c3f7e00949f3b1fb6e8ad5478123', 'request_id': 'req-278f3ef1-d003-43b9-a6eb-b94022085e11'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.923883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c5c3f7e00949f3b1fb6e8ad5478123', 'request_id': 'req-278f3ef1-d003-43b9-a6eb-b94022085e11'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.933667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c5c3f7e00949f3b1fb6e8ad5478123', 'request_id': 'req-278f3ef1-d003-43b9-a6eb-b94022085e11'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.946997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] The token cache is being invalidate because project 25c5c3f7e00949f3b1fb6e8ad5478123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:21.947469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.949200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-278f3ef1-d003-43b9-a6eb-b94022085e11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:21.951789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 833/1675] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:21 2022] DELETE /identity/v3/projects/25c5c3f7e00949f3b1fb6e8ad5478123 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:21.957868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc6c9b-18a1-4d7c-ad3a-e517c162366a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:21.958329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc6c9b-18a1-4d7c-ad3a-e517c162366a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:21.958968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc6c9b-18a1-4d7c-ad3a-e517c162366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:21.988347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-99dc6c9b-18a1-4d7c-ad3a-e517c162366a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:21.989820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-99dc6c9b-18a1-4d7c-ad3a-e517c162366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:22.133737 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 843/1676] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:22.142146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8ca10-ea01-42b2-87a2-2abc42644c23 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.142438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8ca10-ea01-42b2-87a2-2abc42644c23 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.142724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8ca10-ea01-42b2-87a2-2abc42644c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.172493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:22.187783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7ab8ca10-ea01-42b2-87a2-2abc42644c23 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:22.194942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7ab8ca10-ea01-42b2-87a2-2abc42644c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:22.209107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 834/1677] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:22.213245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc6c9b-18a1-4d7c-ad3a-e517c162366a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:22.234925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc6c9b-18a1-4d7c-ad3a-e517c162366a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:22.247587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:22.248470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.248916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.249304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.250082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:22.250583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:22.252366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:22.272522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 844/1678] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:22.273472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8ca10-ea01-42b2-87a2-2abc42644c23 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:22.286186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8ca10-ea01-42b2-87a2-2abc42644c23 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:22.292797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eee544-4419-4e8e-bf4b-54865a1031a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:22.292958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eee544-4419-4e8e-bf4b-54865a1031a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.293226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eee544-4419-4e8e-bf4b-54865a1031a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.293573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eee544-4419-4e8e-bf4b-54865a1031a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.294524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eee544-4419-4e8e-bf4b-54865a1031a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:22.298910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eee544-4419-4e8e-bf4b-54865a1031a7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:22.343728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 835/1679] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:22 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:22.344696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:22.356864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c0472-7eb7-44c2-b1fd-f9e9bcbcf81e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:22.440091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa993d-3219-4e45-816c-2003de1fd578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:22.441015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa993d-3219-4e45-816c-2003de1fd578 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.441509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa993d-3219-4e45-816c-2003de1fd578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.441964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa993d-3219-4e45-816c-2003de1fd578 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.443143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa993d-3219-4e45-816c-2003de1fd578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:22.445683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa993d-3219-4e45-816c-2003de1fd578 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:22.508483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 845/1680] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:22 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:22.513137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eee544-4419-4e8e-bf4b-54865a1031a7 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:22.525033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eee544-4419-4e8e-bf4b-54865a1031a7 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:22.530660 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f09b3-dd37-4c59-a19c-859a5d3a55dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:22.531331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f09b3-dd37-4c59-a19c-859a5d3a55dc admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.531628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f09b3-dd37-4c59-a19c-859a5d3a55dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.532107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56f09b3-dd37-4c59-a19c-859a5d3a55dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.533278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f09b3-dd37-4c59-a19c-859a5d3a55dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:22.535657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56f09b3-dd37-4c59-a19c-859a5d3a55dc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:22.550000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 836/1681] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:22.551686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa993d-3219-4e45-816c-2003de1fd578 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:22.565899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa993d-3219-4e45-816c-2003de1fd578 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:22.572223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a1089-1d9a-43fe-89de-f73cd68517e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:22.573439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1089-1d9a-43fe-89de-f73cd68517e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.573816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1089-1d9a-43fe-89de-f73cd68517e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.574191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a1089-1d9a-43fe-89de-f73cd68517e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.575144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a1089-1d9a-43fe-89de-f73cd68517e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:22.577119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a1089-1d9a-43fe-89de-f73cd68517e5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:22.596186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 846/1682] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:22.597523 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f09b3-dd37-4c59-a19c-859a5d3a55dc admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:22.613875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56f09b3-dd37-4c59-a19c-859a5d3a55dc admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:22.631867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96d616-25a8-4ab2-a288-2de73155bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:22.634332 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96d616-25a8-4ab2-a288-2de73155bdf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.635194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96d616-25a8-4ab2-a288-2de73155bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.636024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96d616-25a8-4ab2-a288-2de73155bdf8 admin admin] PATH_INFO: `/v3/projects/5f270594978e492ab32185f2c0c3114b/users/1e1bfe40238040b19377d00961514de3/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.666636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96d616-25a8-4ab2-a288-2de73155bdf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f270594978e492ab32185f2c0c3114b, user_id=1e1bfe40238040b19377d00961514de3, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:22.675505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee96d616-25a8-4ab2-a288-2de73155bdf8 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:22.690698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 837/1683] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:22 2022] PUT /identity/v3/projects/5f270594978e492ab32185f2c0c3114b/users/1e1bfe40238040b19377d00961514de3/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:22.703085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048b4b-bc69-49b1-ae80-c65d760b627f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.703690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048b4b-bc69-49b1-ae80-c65d760b627f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.704078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d048b4b-bc69-49b1-ae80-c65d760b627f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.739576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6d048b4b-bc69-49b1-ae80-c65d760b627f None None] MFA Rules not processed for user `1e1bfe40238040b19377d00961514de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:22.741082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6d048b4b-bc69-49b1-ae80-c65d760b627f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:22.755130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:22.901373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 847/1684] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:22.902466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96d616-25a8-4ab2-a288-2de73155bdf8 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:22.984813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96d616-25a8-4ab2-a288-2de73155bdf8 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:22.989825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8045-84b3-4f5f-85fe-d78fb0ba5d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:22.990441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8045-84b3-4f5f-85fe-d78fb0ba5d02 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:22.990722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8045-84b3-4f5f-85fe-d78fb0ba5d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:22.991176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3b8045-84b3-4f5f-85fe-d78fb0ba5d02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:22.991835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b8045-84b3-4f5f-85fe-d78fb0ba5d02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:22.994259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3b8045-84b3-4f5f-85fe-d78fb0ba5d02 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.036860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 838/1685] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:22 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:23.037960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d048b4b-bc69-49b1-ae80-c65d760b627f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.050291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d048b4b-bc69-49b1-ae80-c65d760b627f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.124910 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4275-9ab7-4594-9eef-b6ff5c3f6d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.125542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4275-9ab7-4594-9eef-b6ff5c3f6d6d admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.125828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4275-9ab7-4594-9eef-b6ff5c3f6d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.126103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4275-9ab7-4594-9eef-b6ff5c3f6d6d admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.126764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f4275-9ab7-4594-9eef-b6ff5c3f6d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.128731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f4275-9ab7-4594-9eef-b6ff5c3f6d6d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.167235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 848/1686] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:23 2022] POST /identity/v3/users => generated 290 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:23.171250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8045-84b3-4f5f-85fe-d78fb0ba5d02 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.182503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3b8045-84b3-4f5f-85fe-d78fb0ba5d02 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.188268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0497ce-8692-46c3-812c-44d54b99ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.188990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0497ce-8692-46c3-812c-44d54b99ea76 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.189796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0497ce-8692-46c3-812c-44d54b99ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.189796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0497ce-8692-46c3-812c-44d54b99ea76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.190734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0497ce-8692-46c3-812c-44d54b99ea76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.192529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0497ce-8692-46c3-812c-44d54b99ea76 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.205803 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 839/1687] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:23.209924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4275-9ab7-4594-9eef-b6ff5c3f6d6d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.222327 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4275-9ab7-4594-9eef-b6ff5c3f6d6d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.229969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b467ae2-ed9b-4c23-8ddb-bd7249a488cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.229969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467ae2-ed9b-4c23-8ddb-bd7249a488cd admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.230172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467ae2-ed9b-4c23-8ddb-bd7249a488cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.230428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b467ae2-ed9b-4c23-8ddb-bd7249a488cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.231120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b467ae2-ed9b-4c23-8ddb-bd7249a488cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.233040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b467ae2-ed9b-4c23-8ddb-bd7249a488cd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.247773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 849/1688] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:13:23.252727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0497ce-8692-46c3-812c-44d54b99ea76 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.266975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0497ce-8692-46c3-812c-44d54b99ea76 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.273629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1f566-874d-4da2-8a52-ac1a8c278a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.274119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f566-874d-4da2-8a52-ac1a8c278a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.274496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f566-874d-4da2-8a52-ac1a8c278a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.275655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1f566-874d-4da2-8a52-ac1a8c278a86 admin admin] PATH_INFO: `/v3/projects/b76a990d96fe4fde8fa16e8c33906877/users/09fd1014da554f96b9fd78c089e9936c/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.294438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1f566-874d-4da2-8a52-ac1a8c278a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b76a990d96fe4fde8fa16e8c33906877, user_id=09fd1014da554f96b9fd78c089e9936c, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.297296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1f566-874d-4da2-8a52-ac1a8c278a86 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.311688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 840/1689] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:23 2022] PUT /identity/v3/projects/b76a990d96fe4fde8fa16e8c33906877/users/09fd1014da554f96b9fd78c089e9936c/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:23.322062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b427ee-6751-4886-bb84-9b73ee5d096e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.322357 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b427ee-6751-4886-bb84-9b73ee5d096e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.322662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b427ee-6751-4886-bb84-9b73ee5d096e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.351904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-88b427ee-6751-4886-bb84-9b73ee5d096e None None] MFA Rules not processed for user `09fd1014da554f96b9fd78c089e9936c`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:23.353486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-88b427ee-6751-4886-bb84-9b73ee5d096e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:23.428627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 850/1690] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:23.430151 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1f566-874d-4da2-8a52-ac1a8c278a86 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.491831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:23.505790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1f566-874d-4da2-8a52-ac1a8c278a86 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.510865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe90d6-4771-4523-860b-b2d9957011dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.511542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe90d6-4771-4523-860b-b2d9957011dc admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.511827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe90d6-4771-4523-860b-b2d9957011dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.512101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abe90d6-4771-4523-860b-b2d9957011dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.512742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe90d6-4771-4523-860b-b2d9957011dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.514683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abe90d6-4771-4523-860b-b2d9957011dc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.545243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 841/1691] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:23 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:23.546720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b427ee-6751-4886-bb84-9b73ee5d096e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.557593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b427ee-6751-4886-bb84-9b73ee5d096e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.574680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:23.618864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f61332-2f28-4ab2-b617-a0f11efdbfac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.619568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f61332-2f28-4ab2-b617-a0f11efdbfac admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.619934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f61332-2f28-4ab2-b617-a0f11efdbfac admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.620313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f61332-2f28-4ab2-b617-a0f11efdbfac admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.620983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f61332-2f28-4ab2-b617-a0f11efdbfac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.623581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f61332-2f28-4ab2-b617-a0f11efdbfac admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.648225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 851/1692] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:23 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:23.649805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe90d6-4771-4523-860b-b2d9957011dc admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.660062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abe90d6-4771-4523-860b-b2d9957011dc admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.664648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4587-9be5-4ff8-8786-f8fd672c82f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.665247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4587-9be5-4ff8-8786-f8fd672c82f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.665533 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4587-9be5-4ff8-8786-f8fd672c82f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.665810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4587-9be5-4ff8-8786-f8fd672c82f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.666482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df4587-9be5-4ff8-8786-f8fd672c82f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.668100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df4587-9be5-4ff8-8786-f8fd672c82f0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.679686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 842/1693] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:23.683938 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f61332-2f28-4ab2-b617-a0f11efdbfac admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.696201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f61332-2f28-4ab2-b617-a0f11efdbfac admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.702156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dc5e4-a0c9-4eef-83b1-566694301ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.703437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dc5e4-a0c9-4eef-83b1-566694301ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.703917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dc5e4-a0c9-4eef-83b1-566694301ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.704266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dc5e4-a0c9-4eef-83b1-566694301ef7 admin admin] PATH_INFO: `/v3/projects/854f2c8ebb174b9b8ecda35950e6b29f/users/8bf033fe3a6a42c0ae99b587c6e1ad9f/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.722225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069dc5e4-a0c9-4eef-83b1-566694301ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854f2c8ebb174b9b8ecda35950e6b29f, user_id=8bf033fe3a6a42c0ae99b587c6e1ad9f, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.725010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069dc5e4-a0c9-4eef-83b1-566694301ef7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.740316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 852/1694] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:23 2022] PUT /identity/v3/projects/854f2c8ebb174b9b8ecda35950e6b29f/users/8bf033fe3a6a42c0ae99b587c6e1ad9f/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:23.744366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4587-9be5-4ff8-8786-f8fd672c82f0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.756447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4587-9be5-4ff8-8786-f8fd672c82f0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.825421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4529f0d8-da96-4b34-96f0-2ce0780968eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.826082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529f0d8-da96-4b34-96f0-2ce0780968eb admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.826487 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529f0d8-da96-4b34-96f0-2ce0780968eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.826773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4529f0d8-da96-4b34-96f0-2ce0780968eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.827464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529f0d8-da96-4b34-96f0-2ce0780968eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.829097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4529f0d8-da96-4b34-96f0-2ce0780968eb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.839234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 843/1695] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:23.844118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dc5e4-a0c9-4eef-83b1-566694301ef7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:23.854438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069dc5e4-a0c9-4eef-83b1-566694301ef7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:23.859165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:23.859826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.860111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.860393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] PATH_INFO: `/v3/projects/854f2c8ebb174b9b8ecda35950e6b29f/users/8bf033fe3a6a42c0ae99b587c6e1ad9f/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.862302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] RBAC: Authorizing `identity:create_grant(project_id=854f2c8ebb174b9b8ecda35950e6b29f, user_id=8bf033fe3a6a42c0ae99b587c6e1ad9f, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:23.864979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:23.876202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 853/1696] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:23 2022] PUT /identity/v3/projects/854f2c8ebb174b9b8ecda35950e6b29f/users/8bf033fe3a6a42c0ae99b587c6e1ad9f/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:23.887675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d076fe3-4086-4839-bbac-6104ee9f6639 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:23.887968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d076fe3-4086-4839-bbac-6104ee9f6639 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:23.888266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d076fe3-4086-4839-bbac-6104ee9f6639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:23.922083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9d076fe3-4086-4839-bbac-6104ee9f6639 None None] MFA Rules not processed for user `8bf033fe3a6a42c0ae99b587c6e1ad9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:23.923481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9d076fe3-4086-4839-bbac-6104ee9f6639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:24.002485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 844/1697] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:24.014845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:24.017394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:24.083944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4b503-b877-4332-8915-f5f38889a83b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:24.088647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:24.089537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:24.090142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:24.090635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:24.102369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:24.103851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:24.121134 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 854/1698] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:24.168817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d076fe3-4086-4839-bbac-6104ee9f6639 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:24.170747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9d076fe3-4086-4839-bbac-6104ee9f6639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:24.183074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d076fe3-4086-4839-bbac-6104ee9f6639 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:24.187934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faa6cb-3839-4238-8282-059be7aec057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:24.188557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa6cb-3839-4238-8282-059be7aec057 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:24.188895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa6cb-3839-4238-8282-059be7aec057 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:24.189147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faa6cb-3839-4238-8282-059be7aec057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:24.238126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:24.253597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa6cb-3839-4238-8282-059be7aec057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:24.255008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faa6cb-3839-4238-8282-059be7aec057 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:24.265370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 845/1699] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:30.290973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:30.294542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:30.433623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d73cb-41b6-4e49-b97b-c39f85279d15 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:30.444608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:30.445009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:30.445384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:30.445754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] PATH_INFO: `/v3/users/34addecdb9a54cd29874043ea08e4d28` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:30.447009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34addecdb9a54cd29874043ea08e4d28)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:30.450699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:30.472565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:30.520320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34addecdb9a54cd29874043ea08e4d28'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:30.540638 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34addecdb9a54cd29874043ea08e4d28'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:30.542169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34addecdb9a54cd29874043ea08e4d28'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:30.555071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34addecdb9a54cd29874043ea08e4d28'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:30.557320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34addecdb9a54cd29874043ea08e4d28'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:30.575128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 855/1700] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:30 2022] DELETE /identity/v3/users/34addecdb9a54cd29874043ea08e4d28 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:30.581331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ad62-4451-4cdc-a77c-3abf5d92cde0 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:30.582215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ad62-4451-4cdc-a77c-3abf5d92cde0 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:30.582752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96ad62-4451-4cdc-a77c-3abf5d92cde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:30.606556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7f96ad62-4451-4cdc-a77c-3abf5d92cde0 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:30.608752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7f96ad62-4451-4cdc-a77c-3abf5d92cde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:30.690734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 846/1701] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:30.726836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:30.728450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:30.755717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32ee3c-8912-46e1-baab-5582aa7d2788 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:30.756181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32ee3c-8912-46e1-baab-5582aa7d2788 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:30.756518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32ee3c-8912-46e1-baab-5582aa7d2788 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:30.761055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 847/1702] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:13:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:13:30.793052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222f3e-3110-4e9c-951f-c050f9f2ae89 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:30.793891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222f3e-3110-4e9c-951f-c050f9f2ae89 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:30.794172 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8222f3e-3110-4e9c-951f-c050f9f2ae89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:30.820785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a1936-921b-448f-824d-20b9e2f7c197 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:30.825906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:30.826551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:30.826840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:30.827169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:30.845932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a8222f3e-3110-4e9c-951f-c050f9f2ae89 None None] MFA Rules not processed for user `81c4bb19721146c7992d8ea11cd165de`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:30.851818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:30.853493 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:30.855464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a8222f3e-3110-4e9c-951f-c050f9f2ae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:30.866824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 856/1703] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:30.958232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 848/1704] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:13:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:30.971202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:30.972924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:30.991917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcba5d-8cc4-4489-a238-ddd5cf20d99b service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:30.998392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:30.999129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:30.999485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:30.999847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:31.008915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:31.010860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:31.024404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 857/1705] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:31.552555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8222f3e-3110-4e9c-951f-c050f9f2ae89 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:31.554394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a8222f3e-3110-4e9c-951f-c050f9f2ae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:31.574303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8222f3e-3110-4e9c-951f-c050f9f2ae89 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:31.580360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:31.581046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:31.581513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:31.582061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] PATH_INFO: `/v3/users/247fa7c2e727444c9bb969bd6991b7e5` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:31.583430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=247fa7c2e727444c9bb969bd6991b7e5)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:31.585567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:31.652632 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247fa7c2e727444c9bb969bd6991b7e5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.670661 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247fa7c2e727444c9bb969bd6991b7e5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.671871 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247fa7c2e727444c9bb969bd6991b7e5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.682715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247fa7c2e727444c9bb969bd6991b7e5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.691361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:31.694791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247fa7c2e727444c9bb969bd6991b7e5'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.703248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 849/1706] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:31 2022] DELETE /identity/v3/users/247fa7c2e727444c9bb969bd6991b7e5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:31.778008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:31.780156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:31.796080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33ba2-dae1-4c50-b0a2-cee0d6b21506 service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:31.882860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:31.884530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:31.885289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:31.885700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] PATH_INFO: `/v3/users/1e1bfe40238040b19377d00961514de3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:31.886922 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1bfe40238040b19377d00961514de3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:31.889712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:31.921913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:31.948193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1bfe40238040b19377d00961514de3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.967342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1bfe40238040b19377d00961514de3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.970153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1bfe40238040b19377d00961514de3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.993011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1bfe40238040b19377d00961514de3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:31.994815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1bfe40238040b19377d00961514de3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.004199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.006169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.028247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13513512-d152-49e2-a6bd-0a7527366470 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:32.035760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 858/1707] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:31 2022] DELETE /identity/v3/users/1e1bfe40238040b19377d00961514de3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:32.035760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.037241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.041951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:32.042615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.042919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.043336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] PATH_INFO: `/v3/projects/3d7644089a934ed9b9ce5e07345da0c8` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.044406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d7644089a934ed9b9ce5e07345da0c8)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:32.046835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:32.111732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d7644089a934ed9b9ce5e07345da0c8', 'request_id': 'req-7aed40c6-d925-4347-9c65-6b3f693abc27'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.120589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d7644089a934ed9b9ce5e07345da0c8', 'request_id': 'req-7aed40c6-d925-4347-9c65-6b3f693abc27'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.131187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d7644089a934ed9b9ce5e07345da0c8', 'request_id': 'req-7aed40c6-d925-4347-9c65-6b3f693abc27'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.147250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] The token cache is being invalidate because project 3d7644089a934ed9b9ce5e07345da0c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:32.147551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.148222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7aed40c6-d925-4347-9c65-6b3f693abc27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.149058 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca6002-9617-42b3-860a-d155a34ce920 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:32.150557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 850/1708] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:32 2022] DELETE /identity/v3/projects/3d7644089a934ed9b9ce5e07345da0c8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:32.154605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a625-aeee-4f37-8e75-7e5be0d1e629 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.154605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a625-aeee-4f37-8e75-7e5be0d1e629 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.154605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4a625-aeee-4f37-8e75-7e5be0d1e629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.175878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-b4b4a625-aeee-4f37-8e75-7e5be0d1e629 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:32.177432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b4b4a625-aeee-4f37-8e75-7e5be0d1e629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.246738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:32.248146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.248795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.249181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] PATH_INFO: `/v3/projects/42e8ff8265f24a248d5e2f69796e803c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.250751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e8ff8265f24a248d5e2f69796e803c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:32.253507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:32.278736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 851/1709] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:32.281153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c49e-215d-45d4-9618-017d2dc875cd None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.281581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c49e-215d-45d4-9618-017d2dc875cd None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.282228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3c49e-215d-45d4-9618-017d2dc875cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.316724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6ff3c49e-215d-45d4-9618-017d2dc875cd None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:32.318261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6ff3c49e-215d-45d4-9618-017d2dc875cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.329167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e8ff8265f24a248d5e2f69796e803c', 'request_id': 'req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.332335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:32.342055 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e8ff8265f24a248d5e2f69796e803c', 'request_id': 'req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.354383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e8ff8265f24a248d5e2f69796e803c', 'request_id': 'req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.376619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] The token cache is being invalidate because project 42e8ff8265f24a248d5e2f69796e803c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:32.377672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.378580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.380387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 859/1710] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:32 2022] DELETE /identity/v3/projects/42e8ff8265f24a248d5e2f69796e803c => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:32.381251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.390775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.401480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 852/1711] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:32.408363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c49e-215d-45d4-9618-017d2dc875cd None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.410756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6ff3c49e-215d-45d4-9618-017d2dc875cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.428141 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3c49e-215d-45d4-9618-017d2dc875cd None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:32.437354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:32.438281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.438684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.439259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] PATH_INFO: `/v3/users/fbec55028f1d40e09951cb1a76999f28` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.441692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbec55028f1d40e09951cb1a76999f28)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:32.444125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:32.465591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:32.487088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf70cb-4fa2-470a-8eeb-ccabfff9521f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:32.502929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ef4e8-3c44-444d-81cf-9c1b08baa9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:32.503367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ef4e8-3c44-444d-81cf-9c1b08baa9db service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.503640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ef4e8-3c44-444d-81cf-9c1b08baa9db service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.503952 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ef4e8-3c44-444d-81cf-9c1b08baa9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.506118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-589ef4e8-3c44-444d-81cf-9c1b08baa9db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.520177 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ef4e8-3c44-444d-81cf-9c1b08baa9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:32.521659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ef4e8-3c44-444d-81cf-9c1b08baa9db service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:32.523339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbec55028f1d40e09951cb1a76999f28'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.537472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbec55028f1d40e09951cb1a76999f28'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.538579 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbec55028f1d40e09951cb1a76999f28'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.544002 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 860/1712] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:32.546404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eee413-57e9-4ede-8994-750c13c89ba0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.546667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eee413-57e9-4ede-8994-750c13c89ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.549281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eee413-57e9-4ede-8994-750c13c89ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.551235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbec55028f1d40e09951cb1a76999f28'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.561697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbec55028f1d40e09951cb1a76999f28'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:32.574026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 853/1713] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:32 2022] DELETE /identity/v3/users/fbec55028f1d40e09951cb1a76999f28 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:32.585556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f6eee413-57e9-4ede-8994-750c13c89ba0 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:32.587871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f6eee413-57e9-4ede-8994-750c13c89ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.687083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 861/1714] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:32.690959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.704032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39870b-6c42-4a44-8911-9c6991b1b533 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:32.711449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:32.712205 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.712554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.712938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.713665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:32.714178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:32.716392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:32.733577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 854/1715] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:32.738236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eee413-57e9-4ede-8994-750c13c89ba0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.751509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eee413-57e9-4ede-8994-750c13c89ba0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:32.757337 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:32.758329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.758916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.759364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.760653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:32.763512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:32.803188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 862/1716] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:32 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:32.808696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.824849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d519ab-c95d-4334-9ad0-bad86031ce06 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:32.933556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729fdd9-f7ff-436e-8fb2-68b13c300b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:32.934479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729fdd9-f7ff-436e-8fb2-68b13c300b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.934857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729fdd9-f7ff-436e-8fb2-68b13c300b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.935337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729fdd9-f7ff-436e-8fb2-68b13c300b68 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.936517 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729fdd9-f7ff-436e-8fb2-68b13c300b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:32.938728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729fdd9-f7ff-436e-8fb2-68b13c300b68 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:32.945078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.947106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:32.972730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 855/1717] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:32 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:32.976891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729fdd9-f7ff-436e-8fb2-68b13c300b68 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:32.980397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6864ca-a679-405c-8f60-f554b458f20a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:32.989819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:32.990614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:32.991008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:32.991433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] PATH_INFO: `/v3/projects/5f270594978e492ab32185f2c0c3114b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:32.993949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f270594978e492ab32185f2c0c3114b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:32.995774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:32.999847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729fdd9-f7ff-436e-8fb2-68b13c300b68 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:33.014903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad3659-5b87-4b5c-9dc5-5e0445d51306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:33.015959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad3659-5b87-4b5c-9dc5-5e0445d51306 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.016348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad3659-5b87-4b5c-9dc5-5e0445d51306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.017165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad3659-5b87-4b5c-9dc5-5e0445d51306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.017372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad3659-5b87-4b5c-9dc5-5e0445d51306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:33.019260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad3659-5b87-4b5c-9dc5-5e0445d51306 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:33.049937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 856/1718] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:33.054102 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad3659-5b87-4b5c-9dc5-5e0445d51306 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:33.077931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad3659-5b87-4b5c-9dc5-5e0445d51306 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:33.094607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f270594978e492ab32185f2c0c3114b', 'request_id': 'req-798492ff-fbe1-493b-a630-f69441fb9ff7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.116895 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f270594978e492ab32185f2c0c3114b', 'request_id': 'req-798492ff-fbe1-493b-a630-f69441fb9ff7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.132127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:33.141295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f270594978e492ab32185f2c0c3114b', 'request_id': 'req-798492ff-fbe1-493b-a630-f69441fb9ff7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.202971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:33.227358 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] The token cache is being invalidate because project 5f270594978e492ab32185f2c0c3114b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:33.229064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:33.229599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.230369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.231245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.231778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.232704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.233445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:33.235869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:33.237766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 863/1719] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:32 2022] DELETE /identity/v3/projects/5f270594978e492ab32185f2c0c3114b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:33.244729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:33.255325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:33.281160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 857/1720] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:33.285446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798492ff-fbe1-493b-a630-f69441fb9ff7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:33.287643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:33.290311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:33.293158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:33.293715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.295153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.295445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] PATH_INFO: `/v3/projects/cd4daec70cc246f2bde3c88398ad6488` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.296646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd4daec70cc246f2bde3c88398ad6488)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:33.298801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:33.316451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4f8d4-dea7-4923-b34a-c6b199752c34 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:33.338120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:33.338695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.338982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.339291 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] PATH_INFO: `/v3/users/09fd1014da554f96b9fd78c089e9936c` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.340486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=09fd1014da554f96b9fd78c089e9936c)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:33.342798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:33.386061 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd4daec70cc246f2bde3c88398ad6488', 'request_id': 'req-3535c523-77d2-4a21-b453-486924072976'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.394905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd4daec70cc246f2bde3c88398ad6488', 'request_id': 'req-3535c523-77d2-4a21-b453-486924072976'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.407986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd4daec70cc246f2bde3c88398ad6488', 'request_id': 'req-3535c523-77d2-4a21-b453-486924072976'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.440970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] The token cache is being invalidate because project cd4daec70cc246f2bde3c88398ad6488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:33.441372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.447816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.452223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 864/1721] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:33 2022] DELETE /identity/v3/projects/cd4daec70cc246f2bde3c88398ad6488 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:33.453321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:33.456235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:33.464700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09fd1014da554f96b9fd78c089e9936c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.498980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09fd1014da554f96b9fd78c089e9936c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.504082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09fd1014da554f96b9fd78c089e9936c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.517096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09fd1014da554f96b9fd78c089e9936c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.531300 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bf78c366-0dc7-465d-a406-9231c763a4dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09fd1014da554f96b9fd78c089e9936c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:33.547745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 858/1722] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:33 2022] DELETE /identity/v3/users/09fd1014da554f96b9fd78c089e9936c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:33.549092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde68b-c24d-493e-aa8f-448ab9cf752f None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.549385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde68b-c24d-493e-aa8f-448ab9cf752f None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.549665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cde68b-c24d-493e-aa8f-448ab9cf752f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.587456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-87cde68b-c24d-493e-aa8f-448ab9cf752f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:33.588976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-87cde68b-c24d-493e-aa8f-448ab9cf752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:33.620840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535c523-77d2-4a21-b453-486924072976 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:33.682362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 859/1723] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:33.690871 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d464b32-f1be-4c24-b0e3-6496dadf213b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.691199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d464b32-f1be-4c24-b0e3-6496dadf213b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.691732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d464b32-f1be-4c24-b0e3-6496dadf213b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.731751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7d464b32-f1be-4c24-b0e3-6496dadf213b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:33.734383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7d464b32-f1be-4c24-b0e3-6496dadf213b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:33.753354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e8e8e-e5f6-49f4-be88-6f064baad673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:33.754340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8e8e-e5f6-49f4-be88-6f064baad673 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.754628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8e8e-e5f6-49f4-be88-6f064baad673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.754911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8e8e-e5f6-49f4-be88-6f064baad673 admin admin] PATH_INFO: `/v3/projects/50fd5a785c8d45e3bb3d6a9ad90b7137/users/40e043084b794064ba96a27f71d4470b/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.762273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 860/1724] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:33.768621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d464b32-f1be-4c24-b0e3-6496dadf213b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:33.788015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e8e8e-e5f6-49f4-be88-6f064baad673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50fd5a785c8d45e3bb3d6a9ad90b7137, user_id=40e043084b794064ba96a27f71d4470b, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:33.789893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d464b32-f1be-4c24-b0e3-6496dadf213b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:33.790653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e8e8e-e5f6-49f4-be88-6f064baad673 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:33.804546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:33.805192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.806942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.807658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.808340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:33.809802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:33.812553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:33.813451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:33.821586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 865/1725] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:33 2022] PUT /identity/v3/projects/50fd5a785c8d45e3bb3d6a9ad90b7137/users/40e043084b794064ba96a27f71d4470b/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:33.837275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee14fbc-c389-4e77-a34e-a4f1e11c9e37 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:33.837649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee14fbc-c389-4e77-a34e-a4f1e11c9e37 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:33.838125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee14fbc-c389-4e77-a34e-a4f1e11c9e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:33.853294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 861/1726] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:33.858422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:33.877558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38177ad5-eda7-4618-8323-02ebb627f87b admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:33.897962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8ee14fbc-c389-4e77-a34e-a4f1e11c9e37 None None] MFA Rules not processed for user `40e043084b794064ba96a27f71d4470b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:33.906925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8ee14fbc-c389-4e77-a34e-a4f1e11c9e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:33.978135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:34.004999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c5b71-45bc-4b1d-8fde-3c57645a1df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:34.006198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c5b71-45bc-4b1d-8fde-3c57645a1df9 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.008171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c5b71-45bc-4b1d-8fde-3c57645a1df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.008624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c5b71-45bc-4b1d-8fde-3c57645a1df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.009826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c5b71-45bc-4b1d-8fde-3c57645a1df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:34.012253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c5b71-45bc-4b1d-8fde-3c57645a1df9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:34.024917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 866/1727] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 24 13:13:34.036443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee14fbc-c389-4e77-a34e-a4f1e11c9e37 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:34.038898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8ee14fbc-c389-4e77-a34e-a4f1e11c9e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:34.072583 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 862/1728] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:33 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:34.073816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c5b71-45bc-4b1d-8fde-3c57645a1df9 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:34.186834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee14fbc-c389-4e77-a34e-a4f1e11c9e37 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:34.212928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c5b71-45bc-4b1d-8fde-3c57645a1df9 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:34.220045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db2aec-5a4f-46cd-b949-e6d5089c3d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:34.221038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db2aec-5a4f-46cd-b949-e6d5089c3d23 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.221327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db2aec-5a4f-46cd-b949-e6d5089c3d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.221606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db2aec-5a4f-46cd-b949-e6d5089c3d23 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.222319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db2aec-5a4f-46cd-b949-e6d5089c3d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:34.224429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db2aec-5a4f-46cd-b949-e6d5089c3d23 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:34.269234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 863/1729] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:34 2022] POST /identity/v3/users => generated 291 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:34.273700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db2aec-5a4f-46cd-b949-e6d5089c3d23 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:34.296655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db2aec-5a4f-46cd-b949-e6d5089c3d23 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:34.307648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:34.308395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.308753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.309625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.310418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:34.313412 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a875eec-90dc-4970-aa8c-079f37dbbe52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:34.314048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a875eec-90dc-4970-aa8c-079f37dbbe52 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.314331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a875eec-90dc-4970-aa8c-079f37dbbe52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.314607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a875eec-90dc-4970-aa8c-079f37dbbe52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.317634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:34.335139 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 864/1730] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:34.335999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:34.339019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:34.355034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df4ab9b-e00a-4856-8231-f2277d0631e0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:34.360908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:34.361574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.361860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.362144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] PATH_INFO: `/v3/projects/b76a990d96fe4fde8fa16e8c33906877` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.363628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b76a990d96fe4fde8fa16e8c33906877)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:34.365553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:34.428293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a875eec-90dc-4970-aa8c-079f37dbbe52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:34.430870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a875eec-90dc-4970-aa8c-079f37dbbe52 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:34.436609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:34.437014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b76a990d96fe4fde8fa16e8c33906877', 'request_id': 'req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.444107 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 867/1731] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:34.444852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a875eec-90dc-4970-aa8c-079f37dbbe52 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:34.445745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b76a990d96fe4fde8fa16e8c33906877', 'request_id': 'req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.456609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a875eec-90dc-4970-aa8c-079f37dbbe52 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:34.471563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b76a990d96fe4fde8fa16e8c33906877', 'request_id': 'req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.495934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] The token cache is being invalidate because project b76a990d96fe4fde8fa16e8c33906877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:34.496226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.501633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.503729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 865/1732] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:34 2022] DELETE /identity/v3/projects/b76a990d96fe4fde8fa16e8c33906877 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:34.507433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:34.512410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:34.575571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:34.576796 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.577084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.577411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.577874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:34.578009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:34.579810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:34.599507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 868/1733] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:34.604111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:34.607264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:34.624818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124f15-a76c-462f-bf9f-6d128676ecf9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:34.633535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87500441-023c-4216-bd9e-c3f5d5f4b48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:34.636363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87500441-023c-4216-bd9e-c3f5d5f4b48b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.637681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87500441-023c-4216-bd9e-c3f5d5f4b48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.638257 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87500441-023c-4216-bd9e-c3f5d5f4b48b admin admin] PATH_INFO: `/v3/projects/88edf859397a45ba9b2ed76727e5bf7b/users/ab19f6f663e94a87acf19d1ba40992e9/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.659592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab907fc0-6dc4-43d8-8cb8-249b4d2952f0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:34.666840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:34.667545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.667984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.668272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] PATH_INFO: `/v3/users/8bf033fe3a6a42c0ae99b587c6e1ad9f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.670001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf033fe3a6a42c0ae99b587c6e1ad9f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:34.670391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87500441-023c-4216-bd9e-c3f5d5f4b48b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88edf859397a45ba9b2ed76727e5bf7b, user_id=ab19f6f663e94a87acf19d1ba40992e9, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:34.672412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:34.677699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87500441-023c-4216-bd9e-c3f5d5f4b48b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:34.704369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 869/1734] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:34 2022] PUT /identity/v3/projects/88edf859397a45ba9b2ed76727e5bf7b/users/ab19f6f663e94a87acf19d1ba40992e9/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:34.716120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a55f2f-5b11-439a-bd59-b86b66a7e362 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:34.716930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a55f2f-5b11-439a-bd59-b86b66a7e362 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:34.717379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a55f2f-5b11-439a-bd59-b86b66a7e362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:34.790225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-41a55f2f-5b11-439a-bd59-b86b66a7e362 None None] MFA Rules not processed for user `ab19f6f663e94a87acf19d1ba40992e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:34.792907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-41a55f2f-5b11-439a-bd59-b86b66a7e362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:34.795302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf033fe3a6a42c0ae99b587c6e1ad9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.828613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf033fe3a6a42c0ae99b587c6e1ad9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.830427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf033fe3a6a42c0ae99b587c6e1ad9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.844561 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf033fe3a6a42c0ae99b587c6e1ad9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.877883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf033fe3a6a42c0ae99b587c6e1ad9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:34.899490 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 866/1735] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:34 2022] DELETE /identity/v3/users/8bf033fe3a6a42c0ae99b587c6e1ad9f => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:34.982951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 870/1736] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:34.987801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:34.990044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:35.145319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e05f7-61d4-48e0-90f0-916a58d4972d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:35.154463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259eea-812d-490f-bb2e-820af164972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:35.154764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259eea-812d-490f-bb2e-820af164972d admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.155032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259eea-812d-490f-bb2e-820af164972d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.155340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a259eea-812d-490f-bb2e-820af164972d admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.156347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a259eea-812d-490f-bb2e-820af164972d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:35.159543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a259eea-812d-490f-bb2e-820af164972d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:35.204323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 867/1737] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:34 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:35.205095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a55f2f-5b11-439a-bd59-b86b66a7e362 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:35.300790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a55f2f-5b11-439a-bd59-b86b66a7e362 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:35.306207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:35.382431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47539431-0110-45db-ba88-1fadc052e6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:35.382431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47539431-0110-45db-ba88-1fadc052e6ff admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.382431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47539431-0110-45db-ba88-1fadc052e6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.382431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47539431-0110-45db-ba88-1fadc052e6ff admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.383386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47539431-0110-45db-ba88-1fadc052e6ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:35.385372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47539431-0110-45db-ba88-1fadc052e6ff admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:35.429727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 871/1738] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:35 2022] POST /identity/v3/users => generated 299 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:35.433428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259eea-812d-490f-bb2e-820af164972d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:35.452095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a259eea-812d-490f-bb2e-820af164972d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:35.458867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:35.459541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.459829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.460106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.460809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:35.462434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:35.478679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 868/1739] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:35.482878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:35.485288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47539431-0110-45db-ba88-1fadc052e6ff admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:35.488153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:35.504709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47539431-0110-45db-ba88-1fadc052e6ff admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:35.514510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71936-2b6c-438f-a3d2-ca53d9fdbf17 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:35.517389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2345066-04ac-4e2c-883e-f2dd4ff5bbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:35.518082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345066-04ac-4e2c-883e-f2dd4ff5bbfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.518374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345066-04ac-4e2c-883e-f2dd4ff5bbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.518655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2345066-04ac-4e2c-883e-f2dd4ff5bbfb admin admin] PATH_INFO: `/v3/projects/9a89a7d624364f7896a3c85c8011a1ca/users/72f48e73a41e4b998652584bed932886/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.529112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:35.529802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.530084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.530376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] PATH_INFO: `/v3/projects/854f2c8ebb174b9b8ecda35950e6b29f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.532268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=854f2c8ebb174b9b8ecda35950e6b29f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:35.535592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:35.559535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2345066-04ac-4e2c-883e-f2dd4ff5bbfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a89a7d624364f7896a3c85c8011a1ca, user_id=72f48e73a41e4b998652584bed932886, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:35.563730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2345066-04ac-4e2c-883e-f2dd4ff5bbfb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:35.572907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:35.586677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 872/1740] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:35 2022] PUT /identity/v3/projects/9a89a7d624364f7896a3c85c8011a1ca/users/72f48e73a41e4b998652584bed932886/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:35.588063 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2345066-04ac-4e2c-883e-f2dd4ff5bbfb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:35.620390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854f2c8ebb174b9b8ecda35950e6b29f', 'request_id': 'req-3ece7191-43bd-44fc-a68c-30aae43b52c9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:35.629315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854f2c8ebb174b9b8ecda35950e6b29f', 'request_id': 'req-3ece7191-43bd-44fc-a68c-30aae43b52c9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:35.640544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854f2c8ebb174b9b8ecda35950e6b29f', 'request_id': 'req-3ece7191-43bd-44fc-a68c-30aae43b52c9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:35.657589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] The token cache is being invalidate because project 854f2c8ebb174b9b8ecda35950e6b29f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:35.657928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:35.658676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3ece7191-43bd-44fc-a68c-30aae43b52c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:35.660321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 869/1741] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:35 2022] DELETE /identity/v3/projects/854f2c8ebb174b9b8ecda35950e6b29f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:35.666825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0828d-29be-4b27-8017-0ad485be0dbe None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.667589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0828d-29be-4b27-8017-0ad485be0dbe None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.667929 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd0828d-29be-4b27-8017-0ad485be0dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.693600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-abd0828d-29be-4b27-8017-0ad485be0dbe None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:35.694895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-abd0828d-29be-4b27-8017-0ad485be0dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:35.705905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2345066-04ac-4e2c-883e-f2dd4ff5bbfb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:35.788467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 870/1742] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:35.798832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21f07-9dcd-44dd-a2f2-7398cd58d688 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.799397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21f07-9dcd-44dd-a2f2-7398cd58d688 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.799397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab21f07-9dcd-44dd-a2f2-7398cd58d688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.824359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:35.824359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.824359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.824359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.825232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:35.825232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:35.854198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1ab21f07-9dcd-44dd-a2f2-7398cd58d688 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:35.855098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 873/1743] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:35.855444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1ab21f07-9dcd-44dd-a2f2-7398cd58d688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:35.859221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:35.862426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:35.879184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 871/1744] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:35.882767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab21f07-9dcd-44dd-a2f2-7398cd58d688 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:35.893451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57275420-8f33-486d-bfda-0c6160309aed admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:35.900419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab21f07-9dcd-44dd-a2f2-7398cd58d688 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:35.902059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4c10d-de55-4482-b8ad-dca285c672fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:35.902729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4c10d-de55-4482-b8ad-dca285c672fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.904270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4c10d-de55-4482-b8ad-dca285c672fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.904270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4c10d-de55-4482-b8ad-dca285c672fc admin admin] PATH_INFO: `/v3/projects/9a89a7d624364f7896a3c85c8011a1ca/users/72f48e73a41e4b998652584bed932886/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.917508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea636142-416a-492d-956b-6bdef2efcec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:35.918167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4c10d-de55-4482-b8ad-dca285c672fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a89a7d624364f7896a3c85c8011a1ca, user_id=72f48e73a41e4b998652584bed932886, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:35.918302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea636142-416a-492d-956b-6bdef2efcec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.918448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea636142-416a-492d-956b-6bdef2efcec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.918955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea636142-416a-492d-956b-6bdef2efcec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:35.919863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea636142-416a-492d-956b-6bdef2efcec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:35.920172 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea636142-416a-492d-956b-6bdef2efcec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:35.921511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4c10d-de55-4482-b8ad-dca285c672fc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:35.921905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea636142-416a-492d-956b-6bdef2efcec5 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:35.939503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 874/1745] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:35 2022] PUT /identity/v3/projects/9a89a7d624364f7896a3c85c8011a1ca/users/72f48e73a41e4b998652584bed932886/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:35.943337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 872/1746] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:35.943964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4c10d-de55-4482-b8ad-dca285c672fc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:35.956701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:35.959958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63137568-b477-4f50-b9dd-b766b0c2ecb5 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:35.960628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63137568-b477-4f50-b9dd-b766b0c2ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:35.961130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63137568-b477-4f50-b9dd-b766b0c2ecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.001860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-63137568-b477-4f50-b9dd-b766b0c2ecb5 None None] MFA Rules not processed for user `72f48e73a41e4b998652584bed932886`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:36.003566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-63137568-b477-4f50-b9dd-b766b0c2ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:36.095453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4c10d-de55-4482-b8ad-dca285c672fc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:36.101139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f4616-6e39-4226-a442-b1acdae79500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:36.102133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f4616-6e39-4226-a442-b1acdae79500 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.102526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f4616-6e39-4226-a442-b1acdae79500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.102872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f4616-6e39-4226-a442-b1acdae79500 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.103731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f4616-6e39-4226-a442-b1acdae79500 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:36.106107 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f4616-6e39-4226-a442-b1acdae79500 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:36.145137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 873/1747] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:36.157504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63137568-b477-4f50-b9dd-b766b0c2ecb5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:36.159140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-63137568-b477-4f50-b9dd-b766b0c2ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:36.160227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 875/1748] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:35 2022] POST /identity/v3/projects => generated 369 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:36.174224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f4616-6e39-4226-a442-b1acdae79500 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:36.201901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:36.268267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63137568-b477-4f50-b9dd-b766b0c2ecb5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:36.278871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f4616-6e39-4226-a442-b1acdae79500 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:36.285376 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:36.285995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.286278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.286597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.287271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:36.289330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:36.315199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 876/1749] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:36 2022] POST /identity/v3/users => generated 285 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:36.315199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:36.315199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:36.327872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a4a31-1755-4cbf-b143-3fb65dc55e81 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:36.331082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:36.331936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.332355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.332766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.333469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:36.334127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.334819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.335126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:36.335246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] PATH_INFO: `/v3/users/b3e637d319f449f38e4d6375ae5b22fc` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.336761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e637d319f449f38e4d6375ae5b22fc)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:36.339305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:36.400930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e637d319f449f38e4d6375ae5b22fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.415536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:36.417330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:36.420492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e637d319f449f38e4d6375ae5b22fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.421144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e637d319f449f38e4d6375ae5b22fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.429555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e637d319f449f38e4d6375ae5b22fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.430039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 874/1750] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:36.431474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e637d319f449f38e4d6375ae5b22fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.431646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:36.433101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:36.464165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4741f-2fdb-4bf7-ac0c-888c807cef9f service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:36.472563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfcc4-e60a-47cc-92be-8cae35523550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:36.473312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfcc4-e60a-47cc-92be-8cae35523550 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.473732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfcc4-e60a-47cc-92be-8cae35523550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.474108 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83bfcc4-e60a-47cc-92be-8cae35523550 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.477499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 877/1751] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:36 2022] DELETE /identity/v3/users/b3e637d319f449f38e4d6375ae5b22fc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:36.477630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bfcc4-e60a-47cc-92be-8cae35523550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:36.477630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83bfcc4-e60a-47cc-92be-8cae35523550 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:36.491780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 875/1752] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:36.495742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:36.499325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:36.606588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac28bcc-2f0b-4a44-bfcf-999a1a6483bc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:36.615152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:36.616085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.616663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.617021 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.617823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:36.621064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:36.650925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 878/1753] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:13:36.655624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfcc4-e60a-47cc-92be-8cae35523550 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:36.665657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83bfcc4-e60a-47cc-92be-8cae35523550 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:36.679850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d854d98-2e91-47db-ae0f-2c55502831e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:36.680897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d854d98-2e91-47db-ae0f-2c55502831e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.680897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d854d98-2e91-47db-ae0f-2c55502831e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.681116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d854d98-2e91-47db-ae0f-2c55502831e5 admin admin] PATH_INFO: `/v3/projects/57b1e0cfce26405d9ae4182e7acd9a21/users/3326a853fc3a43c283f5750fb9fc650b/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.723028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d854d98-2e91-47db-ae0f-2c55502831e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b1e0cfce26405d9ae4182e7acd9a21, user_id=3326a853fc3a43c283f5750fb9fc650b, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:36.723154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d854d98-2e91-47db-ae0f-2c55502831e5 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:36.736199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:36.739504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:36.766816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 876/1754] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:36 2022] PUT /identity/v3/projects/57b1e0cfce26405d9ae4182e7acd9a21/users/3326a853fc3a43c283f5750fb9fc650b/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:36.777983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09515c44-6aef-41ae-bc4e-521b37e24a28 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:36.778426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10a78-c702-4b52-811b-7423cdb73d6e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.779340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10a78-c702-4b52-811b-7423cdb73d6e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.779627 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb10a78-c702-4b52-811b-7423cdb73d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.829058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-8bb10a78-c702-4b52-811b-7423cdb73d6e None None] MFA Rules not processed for user `3326a853fc3a43c283f5750fb9fc650b`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:36.830559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8bb10a78-c702-4b52-811b-7423cdb73d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:36.867781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:36.868517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.868803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.869299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] PATH_INFO: `/v3/projects/c837e1766e0141ea9b868b89efee5ba1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:36.870466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c837e1766e0141ea9b868b89efee5ba1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:36.872132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:36.880408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:36.915097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 877/1755] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:36.925573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb10a78-c702-4b52-811b-7423cdb73d6e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:36.928833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8bb10a78-c702-4b52-811b-7423cdb73d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:36.951663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c837e1766e0141ea9b868b89efee5ba1', 'request_id': 'req-e35069a9-905b-4c96-ad8b-40ab298535fa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.960873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c837e1766e0141ea9b868b89efee5ba1', 'request_id': 'req-e35069a9-905b-4c96-ad8b-40ab298535fa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.973587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c837e1766e0141ea9b868b89efee5ba1', 'request_id': 'req-e35069a9-905b-4c96-ad8b-40ab298535fa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.988082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] The token cache is being invalidate because project c837e1766e0141ea9b868b89efee5ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:36.988469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.989291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e35069a9-905b-4c96-ad8b-40ab298535fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:36.991460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 879/1756] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:36 2022] DELETE /identity/v3/projects/c837e1766e0141ea9b868b89efee5ba1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:36.996024 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa60c3c-1248-4cfa-9824-0c7a9804fcf7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:36.996315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa60c3c-1248-4cfa-9824-0c7a9804fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:36.996599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa60c3c-1248-4cfa-9824-0c7a9804fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.020611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-baa60c3c-1248-4cfa-9824-0c7a9804fcf7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:37.023198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-baa60c3c-1248-4cfa-9824-0c7a9804fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:37.034164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:37.034972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb10a78-c702-4b52-811b-7423cdb73d6e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:37.089518 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 880/1757] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:37.096632 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d0fc260e254d2d80c6927c52cd8a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:37.097419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.097578 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24287822-2af7-4a22-b90c-3de1f08cae96 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.097993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.098080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24287822-2af7-4a22-b90c-3de1f08cae96 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.098183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.098288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24287822-2af7-4a22-b90c-3de1f08cae96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.099830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:37.118370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-24287822-2af7-4a22-b90c-3de1f08cae96 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:37.119607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-24287822-2af7-4a22-b90c-3de1f08cae96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:37.128316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 881/1758] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:37.132377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24287822-2af7-4a22-b90c-3de1f08cae96 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:37.141822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24287822-2af7-4a22-b90c-3de1f08cae96 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:37.146451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:37.147044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.147360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.147636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.148286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:37.148611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:37.150189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:37.160304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:37.161717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:37.162387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 882/1759] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:13:37.166570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:37.171461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 878/1760] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:37.176947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278e1de-bb36-48d7-8026-b6dbeca40b48 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:37.181677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d9517-c0ee-4248-8605-46fb5fb2cb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:37.182267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d9517-c0ee-4248-8605-46fb5fb2cb4c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.182561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d9517-c0ee-4248-8605-46fb5fb2cb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.182842 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d9517-c0ee-4248-8605-46fb5fb2cb4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.183514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d9517-c0ee-4248-8605-46fb5fb2cb4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:37.186033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d9517-c0ee-4248-8605-46fb5fb2cb4c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:37.217587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 883/1761] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:37 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:37.220905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:37.231799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282608a1-979b-464b-a381-ca8f1af7bf11 service glance] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:37.264947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:37.293835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce00518-05b8-410f-aed5-d79b8a17a284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:37.294949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00518-05b8-410f-aed5-d79b8a17a284 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.295251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00518-05b8-410f-aed5-d79b8a17a284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.295523 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce00518-05b8-410f-aed5-d79b8a17a284 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.296163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce00518-05b8-410f-aed5-d79b8a17a284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:37.298079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce00518-05b8-410f-aed5-d79b8a17a284 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:37.321324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 879/1762] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:37 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:37.325427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d9517-c0ee-4248-8605-46fb5fb2cb4c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:37.337419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d9517-c0ee-4248-8605-46fb5fb2cb4c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:37.342862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289933a0-4a62-4082-ac9f-a0ee5acd6f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:37.343531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289933a0-4a62-4082-ac9f-a0ee5acd6f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.343820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289933a0-4a62-4082-ac9f-a0ee5acd6f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.344096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289933a0-4a62-4082-ac9f-a0ee5acd6f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.344739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289933a0-4a62-4082-ac9f-a0ee5acd6f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:37.346365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289933a0-4a62-4082-ac9f-a0ee5acd6f9e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:37.358941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 884/1763] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:37.362557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce00518-05b8-410f-aed5-d79b8a17a284 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:37.372929 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce00518-05b8-410f-aed5-d79b8a17a284 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:37.377796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1fce0-7295-4f23-9661-2f4c32b5f428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:37.378396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1fce0-7295-4f23-9661-2f4c32b5f428 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.378677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1fce0-7295-4f23-9661-2f4c32b5f428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.378951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1fce0-7295-4f23-9661-2f4c32b5f428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.379630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1fce0-7295-4f23-9661-2f4c32b5f428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:37.381325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1fce0-7295-4f23-9661-2f4c32b5f428 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:37.392673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 880/1764] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:13:37.396850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289933a0-4a62-4082-ac9f-a0ee5acd6f9e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:37.407764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289933a0-4a62-4082-ac9f-a0ee5acd6f9e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:37.412566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:37.413231 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.413516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.413797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] PATH_INFO: `/v3/projects/df0bdf3ab29d43e0bf749b9814813e5c/users/3f69df43e6ca4663a9ef819c0331c13d/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.430437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0bdf3ab29d43e0bf749b9814813e5c, user_id=3f69df43e6ca4663a9ef819c0331c13d, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:37.433169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:37.447442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 885/1765] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:37 2022] PUT /identity/v3/projects/df0bdf3ab29d43e0bf749b9814813e5c/users/3f69df43e6ca4663a9ef819c0331c13d/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:37.452969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e104-1734-4297-a5a3-5e9d993aa62c None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.453329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e104-1734-4297-a5a3-5e9d993aa62c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.453612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e104-1734-4297-a5a3-5e9d993aa62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.475855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e4f5e104-1734-4297-a5a3-5e9d993aa62c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:37.477054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e4f5e104-1734-4297-a5a3-5e9d993aa62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:37.548161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 881/1766] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:37.558184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:37.561260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:37.642756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67e061-5a2c-4e67-ab2b-4c0b4b3eec94 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:37.647072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:37.647758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:37.648255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:37.648378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:37.659467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:37.660917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:37.672726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 886/1767] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:42.250501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a699b-b2e9-45e0-92e6-0414e9070b8d None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:42.251038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a699b-b2e9-45e0-92e6-0414e9070b8d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:42.251038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a699b-b2e9-45e0-92e6-0414e9070b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:42.285092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-fd5a699b-b2e9-45e0-92e6-0414e9070b8d None None] MFA Rules not processed for user `3f69df43e6ca4663a9ef819c0331c13d`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:42.286547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fd5a699b-b2e9-45e0-92e6-0414e9070b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:42.381880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 882/1768] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:42.398125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:42.399684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:42.448515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:42.479960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9eab0-f19d-4a29-922b-acadcb051353 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:42.486301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05652804-882c-4992-aa14-97c33905b1da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:42.486923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05652804-882c-4992-aa14-97c33905b1da service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:42.487239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05652804-882c-4992-aa14-97c33905b1da service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:42.487517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05652804-882c-4992-aa14-97c33905b1da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:42.496055 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05652804-882c-4992-aa14-97c33905b1da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:42.498327 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05652804-882c-4992-aa14-97c33905b1da service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:42.508125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 887/1769] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:44.116877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5ed4-26f8-4fdc-9709-6cd2ea8f9816 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:44.117741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5ed4-26f8-4fdc-9709-6cd2ea8f9816 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:44.118137 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c5ed4-26f8-4fdc-9709-6cd2ea8f9816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:44.140913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-dd8c5ed4-26f8-4fdc-9709-6cd2ea8f9816 None None] MFA Rules not processed for user `3f69df43e6ca4663a9ef819c0331c13d`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:44.142162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-dd8c5ed4-26f8-4fdc-9709-6cd2ea8f9816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:44.150926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 883/1770] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:51.484216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05652804-882c-4992-aa14-97c33905b1da service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:51.485623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-05652804-882c-4992-aa14-97c33905b1da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:51.499688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05652804-882c-4992-aa14-97c33905b1da service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:51.504305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:51.504967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:51.505254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:51.505609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] PATH_INFO: `/v3/users/0cf7d374437a4de8b70ee43bba2ab5bb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:51.506813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf7d374437a4de8b70ee43bba2ab5bb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:51.508946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:51.565480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf7d374437a4de8b70ee43bba2ab5bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:51.588853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf7d374437a4de8b70ee43bba2ab5bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:51.589688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf7d374437a4de8b70ee43bba2ab5bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:51.599977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf7d374437a4de8b70ee43bba2ab5bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:51.600786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf7d374437a4de8b70ee43bba2ab5bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:51.616391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 888/1771] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:51 2022] DELETE /identity/v3/users/0cf7d374437a4de8b70ee43bba2ab5bb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:51.620743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6430f-7527-4f84-83d7-f94fd4dfda37 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:51.621217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6430f-7527-4f84-83d7-f94fd4dfda37 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:51.621297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6430f-7527-4f84-83d7-f94fd4dfda37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:51.649171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-82e6430f-7527-4f84-83d7-f94fd4dfda37 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:51.650393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-82e6430f-7527-4f84-83d7-f94fd4dfda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:51.681657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:51.722684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 884/1772] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:51.733095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:51.734668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:51.827291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d7f58-e77c-4955-a371-a2b0d0960204 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:51.833987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:51.836131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:51.836714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:51.836924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:51.849659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:51.851353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:51.867210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 889/1773] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:51.883255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6430f-7527-4f84-83d7-f94fd4dfda37 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:51.884853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-82e6430f-7527-4f84-83d7-f94fd4dfda37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:51.976879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6430f-7527-4f84-83d7-f94fd4dfda37 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:51.984336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:51.985643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:51.986156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:51.986968 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:51.991235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:52.172441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:52.173891 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:52.188909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 885/1774] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:52.307818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:52.310068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:52.335954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9418c-b49c-42af-87d2-fc607002eabb service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:52.353298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:52.354233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:52.355224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:52.355670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] PATH_INFO: `/v3/projects/ce35ae5204c74b549f2bb6b8f7078e9b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:52.357670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce35ae5204c74b549f2bb6b8f7078e9b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:52.360991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:52.421689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:52.428556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce35ae5204c74b549f2bb6b8f7078e9b', 'request_id': 'req-79159114-7184-4ddc-811c-69fd10382191'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.445657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce35ae5204c74b549f2bb6b8f7078e9b', 'request_id': 'req-79159114-7184-4ddc-811c-69fd10382191'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.455145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce35ae5204c74b549f2bb6b8f7078e9b', 'request_id': 'req-79159114-7184-4ddc-811c-69fd10382191'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.472556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] The token cache is being invalidate because project ce35ae5204c74b549f2bb6b8f7078e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:52.472880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.473861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.478517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 890/1775] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:52 2022] DELETE /identity/v3/projects/ce35ae5204c74b549f2bb6b8f7078e9b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:52.479855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:52.493776 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8edcbf3-5765-4538-9c9b-aa36703f4c6b service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:52.607005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:52.607806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:52.608096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:52.608380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] PATH_INFO: `/v3/users/ccfc7baf62824127ae1924f59c6e0347` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:52.609746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccfc7baf62824127ae1924f59c6e0347)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:52.611854 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:52.664864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:52.710933 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccfc7baf62824127ae1924f59c6e0347'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.728302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccfc7baf62824127ae1924f59c6e0347'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.729114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccfc7baf62824127ae1924f59c6e0347'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.737916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccfc7baf62824127ae1924f59c6e0347'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.745112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-758b581e-fc5f-436c-b7b0-466e8c6b879f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccfc7baf62824127ae1924f59c6e0347'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:52.752318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 886/1776] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:52 2022] DELETE /identity/v3/users/ccfc7baf62824127ae1924f59c6e0347 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:53.041163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:53.042694 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:53.061254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79159114-7184-4ddc-811c-69fd10382191 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:53.129085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:53.129728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.130015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.130299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] PATH_INFO: `/v3/projects/d069e25406b14777975d909a7499ec7a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.131465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d069e25406b14777975d909a7499ec7a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:53.133424 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:53.186229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd069e25406b14777975d909a7499ec7a', 'request_id': 'req-839e3e47-081e-4157-8804-0c3f24dcf7fe'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:53.192849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd069e25406b14777975d909a7499ec7a', 'request_id': 'req-839e3e47-081e-4157-8804-0c3f24dcf7fe'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:53.199729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd069e25406b14777975d909a7499ec7a', 'request_id': 'req-839e3e47-081e-4157-8804-0c3f24dcf7fe'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:53.212447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] The token cache is being invalidate because project d069e25406b14777975d909a7499ec7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:53.212795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:53.213442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-839e3e47-081e-4157-8804-0c3f24dcf7fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:53.214932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 891/1777] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:53 2022] DELETE /identity/v3/projects/d069e25406b14777975d909a7499ec7a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:53.220153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3616b2-ddb9-46a0-acb9-e3b73fcd1443 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.220512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3616b2-ddb9-46a0-acb9-e3b73fcd1443 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.220860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3616b2-ddb9-46a0-acb9-e3b73fcd1443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.242256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-3a3616b2-ddb9-46a0-acb9-e3b73fcd1443 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:53.243616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3a3616b2-ddb9-46a0-acb9-e3b73fcd1443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:53.315937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 887/1778] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:53.324114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db24ce-278f-4a3f-bbd9-0ee9ee107af1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.324405 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db24ce-278f-4a3f-bbd9-0ee9ee107af1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.324682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db24ce-278f-4a3f-bbd9-0ee9ee107af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.344474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b9db24ce-278f-4a3f-bbd9-0ee9ee107af1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:53.345672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b9db24ce-278f-4a3f-bbd9-0ee9ee107af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:53.353641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 892/1779] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:53.358536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3616b2-ddb9-46a0-acb9-e3b73fcd1443 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:53.368227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3616b2-ddb9-46a0-acb9-e3b73fcd1443 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:53.373334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:53.374013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.374298 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.374639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.375936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:53.376263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:53.377970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:53.391632 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 888/1780] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:53.396276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db24ce-278f-4a3f-bbd9-0ee9ee107af1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:53.405728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db24ce-278f-4a3f-bbd9-0ee9ee107af1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:53.410540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d6d79-8c19-43d7-bfc6-31c53a6a0f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:53.411186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d6d79-8c19-43d7-bfc6-31c53a6a0f05 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.411477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d6d79-8c19-43d7-bfc6-31c53a6a0f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.411756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d6d79-8c19-43d7-bfc6-31c53a6a0f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.412401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d6d79-8c19-43d7-bfc6-31c53a6a0f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:53.414323 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d6d79-8c19-43d7-bfc6-31c53a6a0f05 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:53.456524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 893/1781] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:53 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:53.460779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:53.472339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7fc3c-0cb7-4bb2-9e56-aa810cb35757 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:53.555251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6bc84-b63f-4adb-bce8-a11bfb3a61af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:53.555975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6bc84-b63f-4adb-bce8-a11bfb3a61af admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.556323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6bc84-b63f-4adb-bce8-a11bfb3a61af admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.556676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6bc84-b63f-4adb-bce8-a11bfb3a61af admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.557377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6bc84-b63f-4adb-bce8-a11bfb3a61af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:53.559424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6bc84-b63f-4adb-bce8-a11bfb3a61af admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:53.585921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 889/1782] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:53 2022] POST /identity/v3/users => generated 291 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:53.587270 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d6d79-8c19-43d7-bfc6-31c53a6a0f05 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:53.598372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d6d79-8c19-43d7-bfc6-31c53a6a0f05 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:53.603227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44223ba9-1eb8-480f-bda8-8c8f7b44a4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:53.603843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223ba9-1eb8-480f-bda8-8c8f7b44a4eb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.604127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223ba9-1eb8-480f-bda8-8c8f7b44a4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.604405 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44223ba9-1eb8-480f-bda8-8c8f7b44a4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.605239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44223ba9-1eb8-480f-bda8-8c8f7b44a4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:53.607126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44223ba9-1eb8-480f-bda8-8c8f7b44a4eb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:53.619139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 894/1783] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:53.623643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6bc84-b63f-4adb-bce8-a11bfb3a61af admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:53.634642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6bc84-b63f-4adb-bce8-a11bfb3a61af admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:53.640354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ebe76-6716-4775-a722-b126a510627b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:53.640969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ebe76-6716-4775-a722-b126a510627b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.641251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ebe76-6716-4775-a722-b126a510627b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.641526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ebe76-6716-4775-a722-b126a510627b admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.642168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ebe76-6716-4775-a722-b126a510627b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:53.643934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ebe76-6716-4775-a722-b126a510627b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:53.657608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 890/1784] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:53.661712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44223ba9-1eb8-480f-bda8-8c8f7b44a4eb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:53.679546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44223ba9-1eb8-480f-bda8-8c8f7b44a4eb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:53.684367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db15fd0-f827-48ad-9b15-e782233b2888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:53.685672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db15fd0-f827-48ad-9b15-e782233b2888 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.686103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db15fd0-f827-48ad-9b15-e782233b2888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.686541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db15fd0-f827-48ad-9b15-e782233b2888 admin admin] PATH_INFO: `/v3/projects/410f3391a6264d22a6d0a04db7d64eea/users/566e1f9497c241f888117e595c13edd5/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.710548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:53.714872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db15fd0-f827-48ad-9b15-e782233b2888 admin admin] RBAC: Authorizing `identity:create_grant(project_id=410f3391a6264d22a6d0a04db7d64eea, user_id=566e1f9497c241f888117e595c13edd5, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:53.722442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db15fd0-f827-48ad-9b15-e782233b2888 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:53.749403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 895/1785] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:53 2022] PUT /identity/v3/projects/410f3391a6264d22a6d0a04db7d64eea/users/566e1f9497c241f888117e595c13edd5/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:53.761060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c932b8-2ced-4f9b-bae5-cba5745ddb2e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.761851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c932b8-2ced-4f9b-bae5-cba5745ddb2e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.761851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c932b8-2ced-4f9b-bae5-cba5745ddb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.792473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:53.806889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-b8c932b8-2ced-4f9b-bae5-cba5745ddb2e None None] MFA Rules not processed for user `566e1f9497c241f888117e595c13edd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:53.808263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b8c932b8-2ced-4f9b-bae5-cba5745ddb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:53.882623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 891/1786] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:53.888426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db15fd0-f827-48ad-9b15-e782233b2888 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:53.965025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db15fd0-f827-48ad-9b15-e782233b2888 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:53.972094 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bd476-6097-47ba-a769-f71e1f43dc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:53.972789 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bd476-6097-47ba-a769-f71e1f43dc5b admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:53.973252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bd476-6097-47ba-a769-f71e1f43dc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:53.973906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bd476-6097-47ba-a769-f71e1f43dc5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:53.974676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875bd476-6097-47ba-a769-f71e1f43dc5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:53.976701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875bd476-6097-47ba-a769-f71e1f43dc5b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:54.019471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 896/1787] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:53 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:54.020450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c932b8-2ced-4f9b-bae5-cba5745ddb2e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:54.035004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c932b8-2ced-4f9b-bae5-cba5745ddb2e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:54.097218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed69a63-2baf-4869-b7df-ff229315b5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:54.097926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed69a63-2baf-4869-b7df-ff229315b5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:54.098253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed69a63-2baf-4869-b7df-ff229315b5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:54.098627 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed69a63-2baf-4869-b7df-ff229315b5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:54.099418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed69a63-2baf-4869-b7df-ff229315b5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:54.101414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed69a63-2baf-4869-b7df-ff229315b5b1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:54.124711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 892/1788] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:54 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:54.128831 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bd476-6097-47ba-a769-f71e1f43dc5b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:54.140342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bd476-6097-47ba-a769-f71e1f43dc5b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:54.145018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b145a-5b3b-4b39-98af-f1ae58e22833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:54.145644 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b145a-5b3b-4b39-98af-f1ae58e22833 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:54.145925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b145a-5b3b-4b39-98af-f1ae58e22833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:54.146199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b145a-5b3b-4b39-98af-f1ae58e22833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:54.146838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b145a-5b3b-4b39-98af-f1ae58e22833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:54.148461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609b145a-5b3b-4b39-98af-f1ae58e22833 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:54.163142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 897/1789] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:54.167792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed69a63-2baf-4869-b7df-ff229315b5b1 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:54.179362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed69a63-2baf-4869-b7df-ff229315b5b1 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:54.184288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a9d95-1fdb-473d-9542-242d7b938a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:54.184976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a9d95-1fdb-473d-9542-242d7b938a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:54.185260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a9d95-1fdb-473d-9542-242d7b938a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:54.185535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a9d95-1fdb-473d-9542-242d7b938a95 admin admin] PATH_INFO: `/v3/projects/934c94406a9845d2a1d4e19378aecd77/users/e51762aadbe44017878ea3be407f9d94/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:54.202574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a9d95-1fdb-473d-9542-242d7b938a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=934c94406a9845d2a1d4e19378aecd77, user_id=e51762aadbe44017878ea3be407f9d94, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:54.205468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a9d95-1fdb-473d-9542-242d7b938a95 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:54.218921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 893/1790] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:54 2022] PUT /identity/v3/projects/934c94406a9845d2a1d4e19378aecd77/users/e51762aadbe44017878ea3be407f9d94/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:54.223317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b145a-5b3b-4b39-98af-f1ae58e22833 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:54.233407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b145a-5b3b-4b39-98af-f1ae58e22833 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:54.293027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07f2448-d183-4111-a8df-19ab06e97869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:54.293722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f2448-d183-4111-a8df-19ab06e97869 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:54.294232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f2448-d183-4111-a8df-19ab06e97869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:54.294554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07f2448-d183-4111-a8df-19ab06e97869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:54.295324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f2448-d183-4111-a8df-19ab06e97869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:54.297218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07f2448-d183-4111-a8df-19ab06e97869 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:54.308705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 898/1791] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:54.313402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a9d95-1fdb-473d-9542-242d7b938a95 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:54.326218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a9d95-1fdb-473d-9542-242d7b938a95 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:54.332512 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:54.333108 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:54.333385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:54.333648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] PATH_INFO: `/v3/projects/934c94406a9845d2a1d4e19378aecd77/users/e51762aadbe44017878ea3be407f9d94/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:54.335923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=934c94406a9845d2a1d4e19378aecd77, user_id=e51762aadbe44017878ea3be407f9d94, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:54.338576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:54.349836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 894/1792] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:54 2022] PUT /identity/v3/projects/934c94406a9845d2a1d4e19378aecd77/users/e51762aadbe44017878ea3be407f9d94/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:54.362494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c959b-6914-4c8d-a1f9-4bea11344a70 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:54.362838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c959b-6914-4c8d-a1f9-4bea11344a70 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:54.363243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c959b-6914-4c8d-a1f9-4bea11344a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:54.394582 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e38c959b-6914-4c8d-a1f9-4bea11344a70 None None] MFA Rules not processed for user `e51762aadbe44017878ea3be407f9d94`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:54.395841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e38c959b-6914-4c8d-a1f9-4bea11344a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:54.453121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:54.477987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 899/1793] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:54.490702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:54.492405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:54.554115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914ba8e-3d7e-45fe-8aeb-daa5c0c906bc admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:54.557786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842c5f8-c859-4a46-ba1a-d9df42b5b799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:54.558402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c5f8-c859-4a46-ba1a-d9df42b5b799 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:54.558683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c5f8-c859-4a46-ba1a-d9df42b5b799 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:54.558958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842c5f8-c859-4a46-ba1a-d9df42b5b799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:54.567557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:54.575260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842c5f8-c859-4a46-ba1a-d9df42b5b799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:54.577244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842c5f8-c859-4a46-ba1a-d9df42b5b799 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:54.589167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 895/1794] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:54.951571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c959b-6914-4c8d-a1f9-4bea11344a70 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:54.961105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c959b-6914-4c8d-a1f9-4bea11344a70 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:54.966525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e51762aadbe44017878ea3be407f9d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '934c94406a9845d2a1d4e19378aecd77', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:54.966958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:54.967272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:54.967554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:54.968215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:54.968515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:54.970097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:54.982945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 900/1795] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:54.987256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842c5f8-c859-4a46-ba1a-d9df42b5b799 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:54.996287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842c5f8-c859-4a46-ba1a-d9df42b5b799 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:55.001421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752276b-ead4-4d27-9090-c5bc9c00471c tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e51762aadbe44017878ea3be407f9d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '934c94406a9845d2a1d4e19378aecd77', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:55.002024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752276b-ead4-4d27-9090-c5bc9c00471c tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:55.002314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752276b-ead4-4d27-9090-c5bc9c00471c tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:55.002596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752276b-ead4-4d27-9090-c5bc9c00471c tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:55.003254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752276b-ead4-4d27-9090-c5bc9c00471c tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:55.005160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7752276b-ead4-4d27-9090-c5bc9c00471c tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:55.033256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 896/1796] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:54 2022] POST /identity/v3/projects => generated 352 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:55.037418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:55.048154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de3747-19df-418f-aef3-506a3d367e85 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:55.123625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e51762aadbe44017878ea3be407f9d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '934c94406a9845d2a1d4e19378aecd77', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:55.124286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:55.124572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:55.124941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:55.125607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:55.125916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:55.127721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:55.141711 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 901/1797] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:55.188650 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752276b-ead4-4d27-9090-c5bc9c00471c tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:55.199614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752276b-ead4-4d27-9090-c5bc9c00471c tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:55.205224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e51762aadbe44017878ea3be407f9d94', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '934c94406a9845d2a1d4e19378aecd77', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:55.205927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:55.206212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:55.206487 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] PATH_INFO: `/v3/projects/43ac588022624a11a8ed1edc93ead77b` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:55.207629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Authorizing `identity:delete_project(project_id=43ac588022624a11a8ed1edc93ead77b)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:55.209564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:55.257884 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ac588022624a11a8ed1edc93ead77b', 'request_id': 'req-0c7ebe38-9d2b-4100-9740-6840b0d53fae'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.265813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ac588022624a11a8ed1edc93ead77b', 'request_id': 'req-0c7ebe38-9d2b-4100-9740-6840b0d53fae'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.273615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ac588022624a11a8ed1edc93ead77b', 'request_id': 'req-0c7ebe38-9d2b-4100-9740-6840b0d53fae'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.283958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] The token cache is being invalidate because project 43ac588022624a11a8ed1edc93ead77b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:55.284247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.284886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0c7ebe38-9d2b-4100-9740-6840b0d53fae tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.286394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 897/1798] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:55 2022] DELETE /identity/v3/projects/43ac588022624a11a8ed1edc93ead77b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:55.491879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:55.493949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:55.581284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81fc41-70db-4e6c-b5f2-ddb36791e6a4 tempest-VolumeQuotaClassesTest-445551466 tempest-VolumeQuotaClassesTest-445551466-project-admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:55.649078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:55.649741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:55.650090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:55.650449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] PATH_INFO: `/v3/users/566e1f9497c241f888117e595c13edd5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:55.651961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=566e1f9497c241f888117e595c13edd5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:55.654413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:55.707319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:55.774758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '566e1f9497c241f888117e595c13edd5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.801306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '566e1f9497c241f888117e595c13edd5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.804765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '566e1f9497c241f888117e595c13edd5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.818414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '566e1f9497c241f888117e595c13edd5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.819467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '566e1f9497c241f888117e595c13edd5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:55.857913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 902/1799] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:55 2022] DELETE /identity/v3/users/566e1f9497c241f888117e595c13edd5 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:55.863076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d01783-92e7-4b5f-bbe4-04766833ec13 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:55.863830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d01783-92e7-4b5f-bbe4-04766833ec13 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:55.864903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d01783-92e7-4b5f-bbe4-04766833ec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:55.906433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-82d01783-92e7-4b5f-bbe4-04766833ec13 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:55.908516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-82d01783-92e7-4b5f-bbe4-04766833ec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:56.005194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 898/1800] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:56.016221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:56.017631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:56.089286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f995a6fa-1b71-47e9-a7f1-bfacd036d539 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:56.095945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4f1d-17ea-4a99-ba14-380a125d0d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:56.096598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4f1d-17ea-4a99-ba14-380a125d0d4f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:56.096893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4f1d-17ea-4a99-ba14-380a125d0d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:56.097217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4f1d-17ea-4a99-ba14-380a125d0d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:56.111443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4f1d-17ea-4a99-ba14-380a125d0d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:56.112911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4f1d-17ea-4a99-ba14-380a125d0d4f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:56.125218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 903/1801] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:56.474526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d01783-92e7-4b5f-bbe4-04766833ec13 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:56.476169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-82d01783-92e7-4b5f-bbe4-04766833ec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:56.499092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d01783-92e7-4b5f-bbe4-04766833ec13 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:56.504783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:56.505752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:56.506142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:56.506418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] PATH_INFO: `/v3/projects/410f3391a6264d22a6d0a04db7d64eea` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:56.507750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=410f3391a6264d22a6d0a04db7d64eea)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:56.510607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:56.570260 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410f3391a6264d22a6d0a04db7d64eea', 'request_id': 'req-323566f3-e396-451d-99be-8534de40b0f1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.571480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:56.584898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410f3391a6264d22a6d0a04db7d64eea', 'request_id': 'req-323566f3-e396-451d-99be-8534de40b0f1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.595673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410f3391a6264d22a6d0a04db7d64eea', 'request_id': 'req-323566f3-e396-451d-99be-8534de40b0f1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.608651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] The token cache is being invalidate because project 410f3391a6264d22a6d0a04db7d64eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:56.608974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.609827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.611659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 899/1802] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:56 2022] DELETE /identity/v3/projects/410f3391a6264d22a6d0a04db7d64eea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:56.615939 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4f1d-17ea-4a99-ba14-380a125d0d4f service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:56.626414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4f1d-17ea-4a99-ba14-380a125d0d4f service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:56.681826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:56.682387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:56.682671 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:56.682950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] PATH_INFO: `/v3/users/e51762aadbe44017878ea3be407f9d94` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:56.684105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e51762aadbe44017878ea3be407f9d94)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:56.686695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:56.727117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:56.734241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e51762aadbe44017878ea3be407f9d94'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.750385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e51762aadbe44017878ea3be407f9d94'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.751253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e51762aadbe44017878ea3be407f9d94'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.761342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e51762aadbe44017878ea3be407f9d94'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.762379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e51762aadbe44017878ea3be407f9d94'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:56.778680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 904/1803] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:56 2022] DELETE /identity/v3/users/e51762aadbe44017878ea3be407f9d94 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:56.998750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.000220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.013802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323566f3-e396-451d-99be-8534de40b0f1 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.088336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.089031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.089648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.090078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] PATH_INFO: `/v3/users/40e043084b794064ba96a27f71d4470b` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.094759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e043084b794064ba96a27f71d4470b)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.094888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.096247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.097015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.115409 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a53e99-19e8-492c-8a42-81ec8146e7f3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.125848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.126516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.126805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.127202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] PATH_INFO: `/v3/projects/934c94406a9845d2a1d4e19378aecd77` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.128268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=934c94406a9845d2a1d4e19378aecd77)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.130375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.154198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e043084b794064ba96a27f71d4470b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.166994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e043084b794064ba96a27f71d4470b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.167756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e043084b794064ba96a27f71d4470b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.181170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e043084b794064ba96a27f71d4470b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.182978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '934c94406a9845d2a1d4e19378aecd77', 'request_id': 'req-c0a702b3-c47c-415f-8541-d25b04904d8d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.189920 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0d127a59-9fcf-4210-b84d-8d4fd333fde3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e043084b794064ba96a27f71d4470b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.190073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '934c94406a9845d2a1d4e19378aecd77', 'request_id': 'req-c0a702b3-c47c-415f-8541-d25b04904d8d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.197380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 900/1804] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:56 2022] DELETE /identity/v3/users/40e043084b794064ba96a27f71d4470b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:57.201863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '934c94406a9845d2a1d4e19378aecd77', 'request_id': 'req-c0a702b3-c47c-415f-8541-d25b04904d8d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.203378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f48e-e288-4deb-b393-cd13e7f1258b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.203697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f48e-e288-4deb-b393-cd13e7f1258b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.203954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535f48e-e288-4deb-b393-cd13e7f1258b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.217894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] The token cache is being invalidate because project 934c94406a9845d2a1d4e19378aecd77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:57.218192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.218851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c0a702b3-c47c-415f-8541-d25b04904d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.220444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 905/1805] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:57 2022] DELETE /identity/v3/projects/934c94406a9845d2a1d4e19378aecd77 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:57.225027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d880932-9450-44da-a8cb-f9c544832f97 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.225386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1535f48e-e288-4deb-b393-cd13e7f1258b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:57.225527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d880932-9450-44da-a8cb-f9c544832f97 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.225792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d880932-9450-44da-a8cb-f9c544832f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.226580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1535f48e-e288-4deb-b393-cd13e7f1258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.245900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0d880932-9450-44da-a8cb-f9c544832f97 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:57.247646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0d880932-9450-44da-a8cb-f9c544832f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.279244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:57.294792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 901/1806] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:57.304723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f48e-e288-4deb-b393-cd13e7f1258b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.306188 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1535f48e-e288-4deb-b393-cd13e7f1258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.315915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 906/1807] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:57.323298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827c90-afba-4441-ab8f-427ac13ddb66 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.323584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827c90-afba-4441-ab8f-427ac13ddb66 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.323860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81827c90-afba-4441-ab8f-427ac13ddb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.345874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-81827c90-afba-4441-ab8f-427ac13ddb66 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:57.347074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-81827c90-afba-4441-ab8f-427ac13ddb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.358005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 907/1808] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:57.362178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81827c90-afba-4441-ab8f-427ac13ddb66 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.373843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81827c90-afba-4441-ab8f-427ac13ddb66 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.378923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.379596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535f48e-e288-4deb-b393-cd13e7f1258b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.379721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.380001 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.380309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.380960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:57.381302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.382904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.385444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff9661-f8ab-4ac6-8c4b-7be60645bb9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.386824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff9661-f8ab-4ac6-8c4b-7be60645bb9e service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.387268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff9661-f8ab-4ac6-8c4b-7be60645bb9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.387723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff9661-f8ab-4ac6-8c4b-7be60645bb9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.399106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 908/1809] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:13:57.399106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.399364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff9661-f8ab-4ac6-8c4b-7be60645bb9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.399465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.399645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff9661-f8ab-4ac6-8c4b-7be60645bb9e service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.412598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 902/1810] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:57.412598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff9661-f8ab-4ac6-8c4b-7be60645bb9e service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.413820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f5a92-8ed4-493b-930e-c390f6fe5678 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.420856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.421821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.422198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.422456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff9661-f8ab-4ac6-8c4b-7be60645bb9e service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.422654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] PATH_INFO: `/v3/users/ab19f6f663e94a87acf19d1ba40992e9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.423926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab19f6f663e94a87acf19d1ba40992e9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.425936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.427446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96672437-d0e6-4cfe-8c88-d21fb8ced4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.428223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96672437-d0e6-4cfe-8c88-d21fb8ced4e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.428508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96672437-d0e6-4cfe-8c88-d21fb8ced4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.428813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96672437-d0e6-4cfe-8c88-d21fb8ced4e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.429470 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96672437-d0e6-4cfe-8c88-d21fb8ced4e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.431762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96672437-d0e6-4cfe-8c88-d21fb8ced4e6 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.441389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:57.467547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 903/1811] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:57 2022] POST /identity/v3/projects => generated 409 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:57.469482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96672437-d0e6-4cfe-8c88-d21fb8ced4e6 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.500991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab19f6f663e94a87acf19d1ba40992e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.521808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab19f6f663e94a87acf19d1ba40992e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.522576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab19f6f663e94a87acf19d1ba40992e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.534215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab19f6f663e94a87acf19d1ba40992e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.534997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0798fe10-ad38-4857-9604-e85a65dad346 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab19f6f663e94a87acf19d1ba40992e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.550187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96672437-d0e6-4cfe-8c88-d21fb8ced4e6 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.557039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 909/1812] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:57 2022] DELETE /identity/v3/users/ab19f6f663e94a87acf19d1ba40992e9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:57.562237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.562237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.562527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716456-42c9-42a2-b3ec-8a9ae758387e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.562630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.562722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716456-42c9-42a2-b3ec-8a9ae758387e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.562811 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.562907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716456-42c9-42a2-b3ec-8a9ae758387e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.562997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.564259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.588141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-87716456-42c9-42a2-b3ec-8a9ae758387e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:57.589621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-87716456-42c9-42a2-b3ec-8a9ae758387e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.600401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 904/1813] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:57 2022] POST /identity/v3/users => generated 305 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:57.607546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.609384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.630932 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33840471-9aa3-4950-8c72-3d39a2e84866 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.704703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 910/1814] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:57.709946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e867d2-41da-4eee-baba-936a683908a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.711125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e867d2-41da-4eee-baba-936a683908a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.711419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e867d2-41da-4eee-baba-936a683908a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.711699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e867d2-41da-4eee-baba-936a683908a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.712358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e867d2-41da-4eee-baba-936a683908a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.713948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716456-42c9-42a2-b3ec-8a9ae758387e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.714324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e867d2-41da-4eee-baba-936a683908a3 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.715561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-87716456-42c9-42a2-b3ec-8a9ae758387e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.736231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 905/1815] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:57.737997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e867d2-41da-4eee-baba-936a683908a3 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.746143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:57.756060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e867d2-41da-4eee-baba-936a683908a3 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.769665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.770877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.771508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.779592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.780457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.782201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.796597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 906/1816] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:57.801163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.803020 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:57.818897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbf077-578f-4e19-9bc2-71ebec1f2f68 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.820257 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716456-42c9-42a2-b3ec-8a9ae758387e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.824838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5689f-8dea-4267-ad79-d029157edbe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.825618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.826066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5689f-8dea-4267-ad79-d029157edbe2 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.826373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5689f-8dea-4267-ad79-d029157edbe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.826654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5689f-8dea-4267-ad79-d029157edbe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.827304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.827756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.828267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] PATH_INFO: `/v3/projects/50fd5a785c8d45e3bb3d6a9ad90b7137` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.829689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50fd5a785c8d45e3bb3d6a9ad90b7137)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.832204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.839786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5689f-8dea-4267-ad79-d029157edbe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.842801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5689f-8dea-4267-ad79-d029157edbe2 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.855148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 911/1817] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:57.856276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5689f-8dea-4267-ad79-d029157edbe2 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:57.867217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5689f-8dea-4267-ad79-d029157edbe2 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:57.872764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d3a75-e9ab-4b7c-8a76-77d3831b801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:57.873658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3a75-e9ab-4b7c-8a76-77d3831b801d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.874053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3a75-e9ab-4b7c-8a76-77d3831b801d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.874432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d3a75-e9ab-4b7c-8a76-77d3831b801d admin admin] PATH_INFO: `/v3/projects/bba33f1937c447c6a859eef895d33bf1/users/ba12677fbf804c8b8c8b596d79713429/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.897215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d3a75-e9ab-4b7c-8a76-77d3831b801d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bba33f1937c447c6a859eef895d33bf1, user_id=ba12677fbf804c8b8c8b596d79713429, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:57.901218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d3a75-e9ab-4b7c-8a76-77d3831b801d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:57.916889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50fd5a785c8d45e3bb3d6a9ad90b7137', 'request_id': 'req-d592962e-e55b-4392-aea9-c7a6731609a8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.921641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 912/1818] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:57 2022] PUT /identity/v3/projects/bba33f1937c447c6a859eef895d33bf1/users/ba12677fbf804c8b8c8b596d79713429/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:57.935809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428670-a571-4c6c-ba85-a36bf8ad782f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:57.936223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428670-a571-4c6c-ba85-a36bf8ad782f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:57.936746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f428670-a571-4c6c-ba85-a36bf8ad782f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:57.941040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50fd5a785c8d45e3bb3d6a9ad90b7137', 'request_id': 'req-d592962e-e55b-4392-aea9-c7a6731609a8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.956001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50fd5a785c8d45e3bb3d6a9ad90b7137', 'request_id': 'req-d592962e-e55b-4392-aea9-c7a6731609a8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.993578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] The token cache is being invalidate because project 50fd5a785c8d45e3bb3d6a9ad90b7137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:57.994859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:57.995695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:58.000508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.004905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 907/1819] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:57 2022] DELETE /identity/v3/projects/50fd5a785c8d45e3bb3d6a9ad90b7137 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:58.033020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3f428670-a571-4c6c-ba85-a36bf8ad782f None None] MFA Rules not processed for user `ba12677fbf804c8b8c8b596d79713429`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:58.034897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3f428670-a571-4c6c-ba85-a36bf8ad782f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:58.290324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 913/1820] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:57 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:58.295335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:58.297335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:58.399143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592962e-e55b-4392-aea9-c7a6731609a8 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:58.413946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f428670-a571-4c6c-ba85-a36bf8ad782f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:58.415491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3f428670-a571-4c6c-ba85-a36bf8ad782f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:58.498669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f428670-a571-4c6c-ba85-a36bf8ad782f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:58.504233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:58.505181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:58.505288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:58.505570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] PATH_INFO: `/v3/projects/88edf859397a45ba9b2ed76727e5bf7b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:58.506715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88edf859397a45ba9b2ed76727e5bf7b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:58.508747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:58.509625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da13f-9611-4aba-9184-f7d163714ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:58.510594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da13f-9611-4aba-9184-f7d163714ced admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:58.510886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da13f-9611-4aba-9184-f7d163714ced admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:58.511203 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459da13f-9611-4aba-9184-f7d163714ced admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:58.511877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459da13f-9611-4aba-9184-f7d163714ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:58.514526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459da13f-9611-4aba-9184-f7d163714ced admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:58.550187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 908/1821] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:58 2022] POST /identity/v3/projects => generated 409 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:58.556277 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da13f-9611-4aba-9184-f7d163714ced admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:58.566502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:58.570149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88edf859397a45ba9b2ed76727e5bf7b', 'request_id': 'req-338f450b-a462-49c2-9659-070b5cd04a5c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.578140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88edf859397a45ba9b2ed76727e5bf7b', 'request_id': 'req-338f450b-a462-49c2-9659-070b5cd04a5c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.587738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88edf859397a45ba9b2ed76727e5bf7b', 'request_id': 'req-338f450b-a462-49c2-9659-070b5cd04a5c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.601228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] The token cache is being invalidate because project 88edf859397a45ba9b2ed76727e5bf7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:58.601560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.602252 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.607635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 914/1822] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:58 2022] DELETE /identity/v3/projects/88edf859397a45ba9b2ed76727e5bf7b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:58.609253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:58.612243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:58.635394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459da13f-9611-4aba-9184-f7d163714ced admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:58.640929 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:58.641629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:58.641988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:58.642342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:58.643066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:58.645089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:58.655804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:58.674316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 909/1823] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:58 2022] POST /identity/v3/users => generated 311 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:13:58.678565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:58.681019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:58.695434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f450b-a462-49c2-9659-070b5cd04a5c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:58.696094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b074de-c03b-4bb8-836f-ee757a1d5c66 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:58.702155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:58.702804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:58.702915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28609c2-eaea-4f68-ae59-3f34db3b8e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:58.703112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:58.703390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] PATH_INFO: `/v3/users/72f48e73a41e4b998652584bed932886` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:58.703713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28609c2-eaea-4f68-ae59-3f34db3b8e96 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:58.704003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28609c2-eaea-4f68-ae59-3f34db3b8e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:58.704285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28609c2-eaea-4f68-ae59-3f34db3b8e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:58.704546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f48e73a41e4b998652584bed932886)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:58.704945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28609c2-eaea-4f68-ae59-3f34db3b8e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:58.706487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:58.706974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28609c2-eaea-4f68-ae59-3f34db3b8e96 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:58.721434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 910/1824] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:58.726520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28609c2-eaea-4f68-ae59-3f34db3b8e96 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:58.739990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28609c2-eaea-4f68-ae59-3f34db3b8e96 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:58.746343 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:58.747213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:58.747594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:58.748125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] PATH_INFO: `/v3/projects/9b6030a94c614150951bed143a2ff177/users/ed77324562404e25a320148a2f653ad5/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:58.769701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6030a94c614150951bed143a2ff177, user_id=ed77324562404e25a320148a2f653ad5, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:58.772310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f48e73a41e4b998652584bed932886'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.773247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:58.787750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f48e73a41e4b998652584bed932886'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.788541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f48e73a41e4b998652584bed932886'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.791835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 911/1825] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:58 2022] PUT /identity/v3/projects/9b6030a94c614150951bed143a2ff177/users/ed77324562404e25a320148a2f653ad5/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:58.796746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:58.799314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:58.800761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f48e73a41e4b998652584bed932886'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.801600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f48e73a41e4b998652584bed932886'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:58.816356 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 915/1826] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:13:58 2022] DELETE /identity/v3/users/72f48e73a41e4b998652584bed932886 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:58.899198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca533d1-3852-483e-a8a2-466c5275669f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:58.972505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32133898-be98-4c2e-8fc9-92957d9fd1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:58.973156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32133898-be98-4c2e-8fc9-92957d9fd1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:58.973439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32133898-be98-4c2e-8fc9-92957d9fd1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:58.973715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32133898-be98-4c2e-8fc9-92957d9fd1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:58.974363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32133898-be98-4c2e-8fc9-92957d9fd1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:58.976073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32133898-be98-4c2e-8fc9-92957d9fd1ef admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:58.987330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 912/1827] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:13:58.991564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.002988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb50a7-9158-47bc-91ab-37f595280f5a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.008194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.008909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.009747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.010418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] PATH_INFO: `/v3/projects/9b6030a94c614150951bed143a2ff177/users/ed77324562404e25a320148a2f653ad5/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.013094 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6030a94c614150951bed143a2ff177, user_id=ed77324562404e25a320148a2f653ad5, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.015925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.027209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 916/1828] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:58 2022] PUT /identity/v3/projects/9b6030a94c614150951bed143a2ff177/users/ed77324562404e25a320148a2f653ad5/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:59.038655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e423e-12a9-437e-9201-636fd975a9c1 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.038966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e423e-12a9-437e-9201-636fd975a9c1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.039294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e423e-12a9-437e-9201-636fd975a9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.054342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:59.076853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e73e423e-12a9-437e-9201-636fd975a9c1 None None] MFA Rules not processed for user `ed77324562404e25a320148a2f653ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:59.078354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e73e423e-12a9-437e-9201-636fd975a9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:59.158877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 913/1829] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:59 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:59.162889 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.164991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:59.171412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e423e-12a9-437e-9201-636fd975a9c1 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.174031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e73e423e-12a9-437e-9201-636fd975a9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:59.251237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a29806-0b3a-4a01-afdc-954d2304bbeb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.256584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.257398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.257697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.258045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] PATH_INFO: `/v3/projects/9a89a7d624364f7896a3c85c8011a1ca` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.259473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a89a7d624364f7896a3c85c8011a1ca)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.261505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.269576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e423e-12a9-437e-9201-636fd975a9c1 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.273752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d759484-0ba0-4226-baea-0addecbb84f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.274786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d759484-0ba0-4226-baea-0addecbb84f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.275123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d759484-0ba0-4226-baea-0addecbb84f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.275408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d759484-0ba0-4226-baea-0addecbb84f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.287257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d759484-0ba0-4226-baea-0addecbb84f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.288720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d759484-0ba0-4226-baea-0addecbb84f5 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.300950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 914/1830] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:59 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:13:59.313447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:59.321815 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a89a7d624364f7896a3c85c8011a1ca', 'request_id': 'req-8a6e0dcc-5028-41d4-8104-9b931712538e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:59.331450 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a89a7d624364f7896a3c85c8011a1ca', 'request_id': 'req-8a6e0dcc-5028-41d4-8104-9b931712538e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:59.343475 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a89a7d624364f7896a3c85c8011a1ca', 'request_id': 'req-8a6e0dcc-5028-41d4-8104-9b931712538e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:59.363471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] The token cache is being invalidate because project 9a89a7d624364f7896a3c85c8011a1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:13:59.363767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:59.364442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:13:59.370627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 917/1831] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:13:59 2022] DELETE /identity/v3/projects/9a89a7d624364f7896a3c85c8011a1ca => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:13:59.373015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547985b-9881-4eb2-a789-e94fd1ca7e75 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.374261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547985b-9881-4eb2-a789-e94fd1ca7e75 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.374394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547985b-9881-4eb2-a789-e94fd1ca7e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.396454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7547985b-9881-4eb2-a789-e94fd1ca7e75 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:59.397685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7547985b-9881-4eb2-a789-e94fd1ca7e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:59.496566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.498466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:59.507272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 915/1832] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:13:59.516503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ec3af-0b71-429b-9a19-2965f8196951 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.516793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ec3af-0b71-429b-9a19-2965f8196951 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.517072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ec3af-0b71-429b-9a19-2965f8196951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.542544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-791ec3af-0b71-429b-9a19-2965f8196951 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:59.543785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-791ec3af-0b71-429b-9a19-2965f8196951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:59.552956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 916/1833] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:13:59.558097 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ec3af-0b71-429b-9a19-2965f8196951 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.568905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ec3af-0b71-429b-9a19-2965f8196951 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.571908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e0dcc-5028-41d4-8104-9b931712538e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.575722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.576371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.576656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.577004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.578664 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:13:59.581598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.582328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.582611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.582880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.583561 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:13:59.583920 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.585452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.599595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 917/1834] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:13:59.603931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.615944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f17ffc-4944-454e-bcf9-5bf30b492668 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.621231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c38c4-da8b-4827-91ab-6e71601ac96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.622409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c38c4-da8b-4827-91ab-6e71601ac96e admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.623273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c38c4-da8b-4827-91ab-6e71601ac96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.623566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c38c4-da8b-4827-91ab-6e71601ac96e admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.624216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c38c4-da8b-4827-91ab-6e71601ac96e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.626397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28c38c4-da8b-4827-91ab-6e71601ac96e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.642743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:13:59.650370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.652121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.657982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 918/1835] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:13:59 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:59.662083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c38c4-da8b-4827-91ab-6e71601ac96e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.666707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 918/1836] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:13:59 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:13:59.751200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c38c4-da8b-4827-91ab-6e71601ac96e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.756531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e33d-4c32-4ee1-a077-bc034390ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.757358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e33d-4c32-4ee1-a077-bc034390ce7a admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.757771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e33d-4c32-4ee1-a077-bc034390ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.758355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9e33d-4c32-4ee1-a077-bc034390ce7a admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.759464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9e33d-4c32-4ee1-a077-bc034390ce7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.763100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9e33d-4c32-4ee1-a077-bc034390ce7a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.789924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 919/1837] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:13:59 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:13:59.803168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.814453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc38a8-30b9-49c9-a848-a1b19b73d1a7 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.821592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a9339-2ba5-479f-9050-7169c5fab49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.822778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a9339-2ba5-479f-9050-7169c5fab49e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.822778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a9339-2ba5-479f-9050-7169c5fab49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.823226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a9339-2ba5-479f-9050-7169c5fab49e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.824037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a9339-2ba5-479f-9050-7169c5fab49e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.826248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a9339-2ba5-479f-9050-7169c5fab49e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.837413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 919/1838] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:59.842200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e33d-4c32-4ee1-a077-bc034390ce7a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.854963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9e33d-4c32-4ee1-a077-bc034390ce7a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.861289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47fc7d-287d-428d-b47b-99de80093ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.862059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47fc7d-287d-428d-b47b-99de80093ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.862446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47fc7d-287d-428d-b47b-99de80093ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.862831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47fc7d-287d-428d-b47b-99de80093ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.863715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47fc7d-287d-428d-b47b-99de80093ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.865457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47fc7d-287d-428d-b47b-99de80093ba4 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.877807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 920/1839] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:13:59.882305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a9339-2ba5-479f-9050-7169c5fab49e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:13:59.894660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a9339-2ba5-479f-9050-7169c5fab49e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:13:59.899552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4081-bd2a-42c2-a2f0-f8e8404baf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:13:59.900219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4081-bd2a-42c2-a2f0-f8e8404baf60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.900512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4081-bd2a-42c2-a2f0-f8e8404baf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.900791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e4081-bd2a-42c2-a2f0-f8e8404baf60 admin admin] PATH_INFO: `/v3/projects/f581f8894f8e41a0a51c0057217a368a/users/b9a3d53e42584e48b863c31a44da811d/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.916925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4081-bd2a-42c2-a2f0-f8e8404baf60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f581f8894f8e41a0a51c0057217a368a, user_id=b9a3d53e42584e48b863c31a44da811d, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:13:59.920380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e4081-bd2a-42c2-a2f0-f8e8404baf60 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:13:59.934683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 920/1840] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:13:59 2022] PUT /identity/v3/projects/f581f8894f8e41a0a51c0057217a368a/users/b9a3d53e42584e48b863c31a44da811d/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:13:59.947800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28a8ca-299e-433a-b535-993d654d97ed None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:13:59.948399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28a8ca-299e-433a-b535-993d654d97ed None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:13:59.948998 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28a8ca-299e-433a-b535-993d654d97ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:13:59.988679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1e28a8ca-299e-433a-b535-993d654d97ed None None] MFA Rules not processed for user `b9a3d53e42584e48b863c31a44da811d`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:13:59.990292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1e28a8ca-299e-433a-b535-993d654d97ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:00.080434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 921/1841] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:13:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:00.092340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4081-bd2a-42c2-a2f0-f8e8404baf60 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:00.146100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:00.151745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e4081-bd2a-42c2-a2f0-f8e8404baf60 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:00.156505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a48e1-114e-4acf-b40f-21427b30534c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:00.157189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a48e1-114e-4acf-b40f-21427b30534c service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:00.157429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a48e1-114e-4acf-b40f-21427b30534c service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:00.157755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a48e1-114e-4acf-b40f-21427b30534c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:00.166996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a48e1-114e-4acf-b40f-21427b30534c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:00.168457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a48e1-114e-4acf-b40f-21427b30534c service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:00.178479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 921/1842] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:14:00.198463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28a8ca-299e-433a-b535-993d654d97ed None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:00.270591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28a8ca-299e-433a-b535-993d654d97ed None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:00.276387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:00.277236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:00.277620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:00.277999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] PATH_INFO: `/v3/users/b9a3d53e42584e48b863c31a44da811d` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:00.280202 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a3d53e42584e48b863c31a44da811d)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:00.282456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:00.285777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:00.335870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a3d53e42584e48b863c31a44da811d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.349192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a3d53e42584e48b863c31a44da811d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.350000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a3d53e42584e48b863c31a44da811d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.359563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a3d53e42584e48b863c31a44da811d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.367335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a3d53e42584e48b863c31a44da811d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.373958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 922/1843] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:00 2022] DELETE /identity/v3/users/b9a3d53e42584e48b863c31a44da811d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:00.378585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4395-a1c3-4926-81e0-a2efa8f1f494 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:00.379623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4395-a1c3-4926-81e0-a2efa8f1f494 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:00.379983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4395-a1c3-4926-81e0-a2efa8f1f494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:00.400087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dd4b4395-a1c3-4926-81e0-a2efa8f1f494 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:00.401404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dd4b4395-a1c3-4926-81e0-a2efa8f1f494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:00.467155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 922/1844] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:00.477183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:00.478631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:00.546146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc75e33-7667-4a6b-bff8-df9797abd873 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:00.549741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0df3e-8b06-495a-8ddd-d172cea05bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:00.550360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0df3e-8b06-495a-8ddd-d172cea05bee service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:00.550643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0df3e-8b06-495a-8ddd-d172cea05bee service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:00.550921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0df3e-8b06-495a-8ddd-d172cea05bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:00.560978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0df3e-8b06-495a-8ddd-d172cea05bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:00.562464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0df3e-8b06-495a-8ddd-d172cea05bee service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:00.572653 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 923/1845] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:00.856901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b4395-a1c3-4926-81e0-a2efa8f1f494 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:00.860455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dd4b4395-a1c3-4926-81e0-a2efa8f1f494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:00.885380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b4395-a1c3-4926-81e0-a2efa8f1f494 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:00.894309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:00.895040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:00.895427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:00.895793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] PATH_INFO: `/v3/projects/f581f8894f8e41a0a51c0057217a368a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:00.897092 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f581f8894f8e41a0a51c0057217a368a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:00.899204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:00.955175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f581f8894f8e41a0a51c0057217a368a', 'request_id': 'req-60e94c87-6f5c-4532-8605-eab417b00617'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.962245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f581f8894f8e41a0a51c0057217a368a', 'request_id': 'req-60e94c87-6f5c-4532-8605-eab417b00617'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.972429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f581f8894f8e41a0a51c0057217a368a', 'request_id': 'req-60e94c87-6f5c-4532-8605-eab417b00617'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.990378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] The token cache is being invalidate because project f581f8894f8e41a0a51c0057217a368a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:00.990678 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.991484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-60e94c87-6f5c-4532-8605-eab417b00617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:00.993194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 923/1846] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:00 2022] DELETE /identity/v3/projects/f581f8894f8e41a0a51c0057217a368a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:00.997866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2a6e8-d819-492f-9f1b-e9a13f16a032 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:00.998158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2a6e8-d819-492f-9f1b-e9a13f16a032 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:00.998435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2a6e8-d819-492f-9f1b-e9a13f16a032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.018733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-70d2a6e8-d819-492f-9f1b-e9a13f16a032 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:01.019982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-70d2a6e8-d819-492f-9f1b-e9a13f16a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:01.111741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 924/1847] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:01.116319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1580bd7-2f7e-4e26-b803-239c1e8368c1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.116319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1580bd7-2f7e-4e26-b803-239c1e8368c1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.116319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1580bd7-2f7e-4e26-b803-239c1e8368c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.150238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c1580bd7-2f7e-4e26-b803-239c1e8368c1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:01.152269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c1580bd7-2f7e-4e26-b803-239c1e8368c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:01.161585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 924/1848] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:01.165900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2a6e8-d819-492f-9f1b-e9a13f16a032 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:01.177129 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2a6e8-d819-492f-9f1b-e9a13f16a032 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:01.185329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:01.185953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.189226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.189226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.189657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:14:01.190130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:01.192996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:01.217571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1580bd7-2f7e-4e26-b803-239c1e8368c1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:01.225169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 925/1849] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:14:01.230900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1580bd7-2f7e-4e26-b803-239c1e8368c1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:01.238068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfc6e1-4fa7-4117-bb6a-7703e2c2bc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:01.239528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfc6e1-4fa7-4117-bb6a-7703e2c2bc3a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.240044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfc6e1-4fa7-4117-bb6a-7703e2c2bc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.240512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfc6e1-4fa7-4117-bb6a-7703e2c2bc3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.241415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfc6e1-4fa7-4117-bb6a-7703e2c2bc3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:01.243838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfc6e1-4fa7-4117-bb6a-7703e2c2bc3a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:01.283130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 925/1850] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:14:01 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:01.286413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:01.302922 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6703430-4a24-423a-aff6-7566f0a7a5e6 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:01.323325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:01.412652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234594f-a2e9-4561-9af1-282697c05027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:01.413375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234594f-a2e9-4561-9af1-282697c05027 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.413375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234594f-a2e9-4561-9af1-282697c05027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.413648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234594f-a2e9-4561-9af1-282697c05027 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.414289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234594f-a2e9-4561-9af1-282697c05027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:01.416434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234594f-a2e9-4561-9af1-282697c05027 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:01.445638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 926/1851] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:14:01 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:01.446883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfc6e1-4fa7-4117-bb6a-7703e2c2bc3a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:01.457987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfc6e1-4fa7-4117-bb6a-7703e2c2bc3a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:01.463606 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d1698-16c2-42f2-bf43-6d283c7b471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:01.464364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1698-16c2-42f2-bf43-6d283c7b471b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.464747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1698-16c2-42f2-bf43-6d283c7b471b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.465126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d1698-16c2-42f2-bf43-6d283c7b471b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.466460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d1698-16c2-42f2-bf43-6d283c7b471b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:01.467930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d1698-16c2-42f2-bf43-6d283c7b471b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:01.482500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 926/1852] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:01.483846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234594f-a2e9-4561-9af1-282697c05027 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:01.494503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234594f-a2e9-4561-9af1-282697c05027 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:01.499476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbdc00-ab53-4cb3-9781-b3770b081c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:01.500092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbdc00-ab53-4cb3-9781-b3770b081c41 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.500378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbdc00-ab53-4cb3-9781-b3770b081c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.500657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbdc00-ab53-4cb3-9781-b3770b081c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.501310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbdc00-ab53-4cb3-9781-b3770b081c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:01.502953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbdc00-ab53-4cb3-9781-b3770b081c41 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:01.516227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 927/1853] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:01.520309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d1698-16c2-42f2-bf43-6d283c7b471b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:01.531321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d1698-16c2-42f2-bf43-6d283c7b471b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:01.536263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:01.537012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.537652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.537652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] PATH_INFO: `/v3/projects/5b028ba991e14292803f73d38f7003dc/users/1fd3d61c9fad4ac8ae26433ec77cd9c8/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.554364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b028ba991e14292803f73d38f7003dc, user_id=1fd3d61c9fad4ac8ae26433ec77cd9c8, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:01.557241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:01.573702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 927/1854] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:14:01 2022] PUT /identity/v3/projects/5b028ba991e14292803f73d38f7003dc/users/1fd3d61c9fad4ac8ae26433ec77cd9c8/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:01.579477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af7be9-d276-4ab6-840b-a5e0d30d2bd8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.579840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af7be9-d276-4ab6-840b-a5e0d30d2bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.580122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af7be9-d276-4ab6-840b-a5e0d30d2bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.601034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f7af7be9-d276-4ab6-840b-a5e0d30d2bd8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:01.602257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f7af7be9-d276-4ab6-840b-a5e0d30d2bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:01.674366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 928/1855] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:01.684114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:01.686831 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:01.697947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:01.748988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b156df06-890d-4123-86ad-610f411c55bc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:01.752942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:01.753560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:01.753844 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:01.754120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:01.767009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:01.768580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:01.779379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 928/1856] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:02.523011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfd6f9-af73-4aa8-a1b6-1439428e814b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:02.523566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfd6f9-af73-4aa8-a1b6-1439428e814b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:02.523666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfd6f9-af73-4aa8-a1b6-1439428e814b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:02.552710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-debfd6f9-af73-4aa8-a1b6-1439428e814b None None] MFA Rules not processed for user `1fd3d61c9fad4ac8ae26433ec77cd9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:02.554071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-debfd6f9-af73-4aa8-a1b6-1439428e814b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:02.620929 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 929/1857] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:02.632176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:02.633604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:02.693262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a964aee-1b40-4994-8b20-c4a062f05812 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:02.696962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:02.697682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:02.697851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:02.698238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:02.707298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:02.708733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:02.717802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 929/1858] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:05.131392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46215c-554e-480d-86d0-ac3923cc9125 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:05.131890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46215c-554e-480d-86d0-ac3923cc9125 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:05.131981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46215c-554e-480d-86d0-ac3923cc9125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:05.152719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5c46215c-554e-480d-86d0-ac3923cc9125 None None] MFA Rules not processed for user `1fd3d61c9fad4ac8ae26433ec77cd9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:05.153993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5c46215c-554e-480d-86d0-ac3923cc9125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:05.160774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 930/1859] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:08.367621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:08.369114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:08.383625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df5439-bf9a-467e-8406-e0cf2c787e0b service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:08.388431 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:08.389052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:08.389341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:08.389631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] PATH_INFO: `/v3/users/ba12677fbf804c8b8c8b596d79713429` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:08.390759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba12677fbf804c8b8c8b596d79713429)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:08.392725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:08.439675 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba12677fbf804c8b8c8b596d79713429'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:08.451548 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba12677fbf804c8b8c8b596d79713429'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:08.452223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba12677fbf804c8b8c8b596d79713429'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:08.459607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba12677fbf804c8b8c8b596d79713429'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:08.460227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba12677fbf804c8b8c8b596d79713429'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:08.473632 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 930/1860] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:08 2022] DELETE /identity/v3/users/ba12677fbf804c8b8c8b596d79713429 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:08.478099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387d881-e930-48e4-bd12-1423f28fd9c5 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:08.478546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387d881-e930-48e4-bd12-1423f28fd9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:08.478655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387d881-e930-48e4-bd12-1423f28fd9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:08.498145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7387d881-e930-48e4-bd12-1423f28fd9c5 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:08.499365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7387d881-e930-48e4-bd12-1423f28fd9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:08.507549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:08.560293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 931/1861] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:08.569800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:08.571216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:08.602027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:08.636885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6ee02-65f9-4384-ac44-3ac8a5aa97b1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:08.640662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97c6d-b029-43e5-afb9-e30d61be1c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:08.641269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97c6d-b029-43e5-afb9-e30d61be1c6d service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:08.641549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97c6d-b029-43e5-afb9-e30d61be1c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:08.641825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97c6d-b029-43e5-afb9-e30d61be1c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:08.650836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c97c6d-b029-43e5-afb9-e30d61be1c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:08.652285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c97c6d-b029-43e5-afb9-e30d61be1c6d service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:08.662225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 931/1862] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:08.930901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387d881-e930-48e4-bd12-1423f28fd9c5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:08.932458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7387d881-e930-48e4-bd12-1423f28fd9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:08.948615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387d881-e930-48e4-bd12-1423f28fd9c5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:08.954714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:08.955392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:08.955678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:08.955962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] PATH_INFO: `/v3/projects/bba33f1937c447c6a859eef895d33bf1` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:08.957068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bba33f1937c447c6a859eef895d33bf1)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:08.958977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:09.006071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bba33f1937c447c6a859eef895d33bf1', 'request_id': 'req-7140382f-6b6a-4716-8e8f-86182762c62d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.013031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bba33f1937c447c6a859eef895d33bf1', 'request_id': 'req-7140382f-6b6a-4716-8e8f-86182762c62d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.020191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bba33f1937c447c6a859eef895d33bf1', 'request_id': 'req-7140382f-6b6a-4716-8e8f-86182762c62d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.031650 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] The token cache is being invalidate because project bba33f1937c447c6a859eef895d33bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:09.031937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.032553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.034029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 932/1863] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:08 2022] DELETE /identity/v3/projects/bba33f1937c447c6a859eef895d33bf1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:09.038103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97c6d-b029-43e5-afb9-e30d61be1c6d service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:09.047847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97c6d-b029-43e5-afb9-e30d61be1c6d service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:09.108283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:09.108927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:09.109228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:09.109515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] PATH_INFO: `/v3/users/ed77324562404e25a320148a2f653ad5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:09.110676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed77324562404e25a320148a2f653ad5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:09.112680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:09.165498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed77324562404e25a320148a2f653ad5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.181086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed77324562404e25a320148a2f653ad5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.182008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed77324562404e25a320148a2f653ad5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.192229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed77324562404e25a320148a2f653ad5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.199251 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-770221d4-6571-4482-9730-41999fbd0f05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed77324562404e25a320148a2f653ad5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.215639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 932/1864] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:09 2022] DELETE /identity/v3/users/ed77324562404e25a320148a2f653ad5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:09.576094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:09.577540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:09.591144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140382f-6b6a-4716-8e8f-86182762c62d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:09.650989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:09.651656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:09.651940 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:09.652216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] PATH_INFO: `/v3/projects/9b6030a94c614150951bed143a2ff177` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:09.653351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6030a94c614150951bed143a2ff177)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:09.655288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:09.694302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:09.702425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6030a94c614150951bed143a2ff177', 'request_id': 'req-10510b02-8b36-4da1-9c4e-3297733e2cda'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.708702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6030a94c614150951bed143a2ff177', 'request_id': 'req-10510b02-8b36-4da1-9c4e-3297733e2cda'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.715683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6030a94c614150951bed143a2ff177', 'request_id': 'req-10510b02-8b36-4da1-9c4e-3297733e2cda'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.736448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] The token cache is being invalidate because project 9b6030a94c614150951bed143a2ff177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:09.736737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.737384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-10510b02-8b36-4da1-9c4e-3297733e2cda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:09.738866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 933/1865] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:09 2022] DELETE /identity/v3/projects/9b6030a94c614150951bed143a2ff177 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:09.743658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6384d5-03b1-4f5e-b969-53d3518ebc94 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:09.744162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6384d5-03b1-4f5e-b969-53d3518ebc94 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:09.744259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6384d5-03b1-4f5e-b969-53d3518ebc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:09.767508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7c6384d5-03b1-4f5e-b969-53d3518ebc94 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:09.768711 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7c6384d5-03b1-4f5e-b969-53d3518ebc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:09.811986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:09.830848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 933/1866] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:09.838598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd34c02-6f8a-4b49-a6c0-16d2b29b714b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:09.838892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd34c02-6f8a-4b49-a6c0-16d2b29b714b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:09.839223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd34c02-6f8a-4b49-a6c0-16d2b29b714b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:09.859413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5cd34c02-6f8a-4b49-a6c0-16d2b29b714b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:09.860616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5cd34c02-6f8a-4b49-a6c0-16d2b29b714b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:09.868346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 934/1867] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:09.872973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6384d5-03b1-4f5e-b969-53d3518ebc94 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:09.882482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6384d5-03b1-4f5e-b969-53d3518ebc94 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:09.887982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:09.888752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:09.889032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:09.889336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:09.889981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:14:09.890287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:09.891945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:09.904122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 934/1868] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:14:09.908286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd34c02-6f8a-4b49-a6c0-16d2b29b714b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:09.917413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd34c02-6f8a-4b49-a6c0-16d2b29b714b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:09.922255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a953c1-e024-451e-9bf5-45c545a0d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:09.922859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a953c1-e024-451e-9bf5-45c545a0d79e admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:09.923245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a953c1-e024-451e-9bf5-45c545a0d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:09.923527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a953c1-e024-451e-9bf5-45c545a0d79e admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:09.924163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a953c1-e024-451e-9bf5-45c545a0d79e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:09.926061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a953c1-e024-451e-9bf5-45c545a0d79e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:09.954021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 935/1869] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:14:09 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:09.958147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:09.968060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d83d76-2ee2-4dfe-ae23-990314070fd7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:10.024933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c8ff0-a45f-4e9a-8292-47558efd905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:10.025524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c8ff0-a45f-4e9a-8292-47558efd905f admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.025802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c8ff0-a45f-4e9a-8292-47558efd905f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.026075 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c8ff0-a45f-4e9a-8292-47558efd905f admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.026709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c8ff0-a45f-4e9a-8292-47558efd905f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:10.029367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c8ff0-a45f-4e9a-8292-47558efd905f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:10.050768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 935/1870] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:14:09 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:14:10.055002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a953c1-e024-451e-9bf5-45c545a0d79e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:10.065688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a953c1-e024-451e-9bf5-45c545a0d79e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:10.070443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9a3a-d7a2-4847-bb5b-caa7391da465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:10.071098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9a3a-d7a2-4847-bb5b-caa7391da465 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.071390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9a3a-d7a2-4847-bb5b-caa7391da465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.071670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856f9a3a-d7a2-4847-bb5b-caa7391da465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.072321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f9a3a-d7a2-4847-bb5b-caa7391da465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:10.074069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856f9a3a-d7a2-4847-bb5b-caa7391da465 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:10.085091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 936/1871] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:10.089139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c8ff0-a45f-4e9a-8292-47558efd905f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:10.099605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c8ff0-a45f-4e9a-8292-47558efd905f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:10.104170 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7cb69-1b99-49de-b48c-fa8a40f0aa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:10.104759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7cb69-1b99-49de-b48c-fa8a40f0aa0b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.105087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7cb69-1b99-49de-b48c-fa8a40f0aa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.105487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7cb69-1b99-49de-b48c-fa8a40f0aa0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.106772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7cb69-1b99-49de-b48c-fa8a40f0aa0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:10.108365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7cb69-1b99-49de-b48c-fa8a40f0aa0b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:10.118159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 936/1872] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:10.122356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9a3a-d7a2-4847-bb5b-caa7391da465 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:10.132780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856f9a3a-d7a2-4847-bb5b-caa7391da465 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:10.137749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:10.138504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.138790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.139091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] PATH_INFO: `/v3/projects/fcd8c04db38e4ad9a2375523a3e083ee/users/3cd8546547ae4f3bb9618d86f5b90cc4/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.154836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd8c04db38e4ad9a2375523a3e083ee, user_id=3cd8546547ae4f3bb9618d86f5b90cc4, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:10.157700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:10.172667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 937/1873] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:14:10 2022] PUT /identity/v3/projects/fcd8c04db38e4ad9a2375523a3e083ee/users/3cd8546547ae4f3bb9618d86f5b90cc4/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:10.183834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab358-054f-4430-9cd0-55d4dc043280 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.184169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab358-054f-4430-9cd0-55d4dc043280 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.184446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab358-054f-4430-9cd0-55d4dc043280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.213716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-317ab358-054f-4430-9cd0-55d4dc043280 None None] MFA Rules not processed for user `3cd8546547ae4f3bb9618d86f5b90cc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:10.214929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-317ab358-054f-4430-9cd0-55d4dc043280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:10.295614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 937/1874] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:10.307086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:10.308996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:10.370562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89e1fd-d6d1-4984-a160-ee03283333e9 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:10.374888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85333792-019d-418b-955e-0bb256f59e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:10.375598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333792-019d-418b-955e-0bb256f59e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.375969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333792-019d-418b-955e-0bb256f59e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.376314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85333792-019d-418b-955e-0bb256f59e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.384897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85333792-019d-418b-955e-0bb256f59e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:10.386588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85333792-019d-418b-955e-0bb256f59e7a service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:10.396375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 938/1875] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:10.421662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ab358-054f-4430-9cd0-55d4dc043280 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:10.472666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:10.491794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ab358-054f-4430-9cd0-55d4dc043280 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:10.498017 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:10.499085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.499085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.499313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] PATH_INFO: `/v3/users/3cd8546547ae4f3bb9618d86f5b90cc4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.500770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd8546547ae4f3bb9618d86f5b90cc4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:10.502768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:10.552937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd8546547ae4f3bb9618d86f5b90cc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:10.566519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd8546547ae4f3bb9618d86f5b90cc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:10.567410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd8546547ae4f3bb9618d86f5b90cc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:10.576650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd8546547ae4f3bb9618d86f5b90cc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:10.577415 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd8546547ae4f3bb9618d86f5b90cc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:10.607194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 938/1876] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:10 2022] DELETE /identity/v3/users/3cd8546547ae4f3bb9618d86f5b90cc4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:10.613816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31e478-93e8-4c35-bd59-bbcdbd56a288 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.614311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31e478-93e8-4c35-bd59-bbcdbd56a288 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.614438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31e478-93e8-4c35-bd59-bbcdbd56a288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.638705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5d31e478-93e8-4c35-bd59-bbcdbd56a288 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:10.639994 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5d31e478-93e8-4c35-bd59-bbcdbd56a288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:10.709801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 939/1877] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:10.728989 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:10.730847 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:10.799120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eb5c1-1dae-4a9a-a89b-af731b99f50e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:10.802850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def22f6-bab8-4222-836d-0b2097bc46c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:10.803640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def22f6-bab8-4222-836d-0b2097bc46c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:10.803922 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def22f6-bab8-4222-836d-0b2097bc46c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:10.804199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def22f6-bab8-4222-836d-0b2097bc46c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:10.814385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def22f6-bab8-4222-836d-0b2097bc46c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:10.815907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def22f6-bab8-4222-836d-0b2097bc46c1 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:10.827041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 939/1878] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:11.427372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31e478-93e8-4c35-bd59-bbcdbd56a288 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:11.429379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5d31e478-93e8-4c35-bd59-bbcdbd56a288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:11.443691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31e478-93e8-4c35-bd59-bbcdbd56a288 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:11.449074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:11.450729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:11.451433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:11.451738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] PATH_INFO: `/v3/projects/fcd8c04db38e4ad9a2375523a3e083ee` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:11.453111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd8c04db38e4ad9a2375523a3e083ee)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:11.455411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:11.457119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:11.511285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd8c04db38e4ad9a2375523a3e083ee', 'request_id': 'req-19043f0c-a996-4015-9865-5b30fe6dfbce'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:11.518477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd8c04db38e4ad9a2375523a3e083ee', 'request_id': 'req-19043f0c-a996-4015-9865-5b30fe6dfbce'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:11.535701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd8c04db38e4ad9a2375523a3e083ee', 'request_id': 'req-19043f0c-a996-4015-9865-5b30fe6dfbce'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:11.549247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] The token cache is being invalidate because project fcd8c04db38e4ad9a2375523a3e083ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:11.549619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:11.551184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-19043f0c-a996-4015-9865-5b30fe6dfbce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:11.553157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 940/1879] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:11 2022] DELETE /identity/v3/projects/fcd8c04db38e4ad9a2375523a3e083ee => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:11.557782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded53e2-2b85-447b-b482-6627dda1272f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:11.558080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded53e2-2b85-447b-b482-6627dda1272f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:11.560392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded53e2-2b85-447b-b482-6627dda1272f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:11.580332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9ded53e2-2b85-447b-b482-6627dda1272f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:11.582000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9ded53e2-2b85-447b-b482-6627dda1272f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:11.659586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 940/1880] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:14:11.662384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095191e8-c5f7-4ad5-945d-d41d29f501df None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:11.662879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095191e8-c5f7-4ad5-945d-d41d29f501df None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:11.662961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095191e8-c5f7-4ad5-945d-d41d29f501df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:11.685963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-095191e8-c5f7-4ad5-945d-d41d29f501df None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:11.686882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-095191e8-c5f7-4ad5-945d-d41d29f501df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:11.695674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 941/1881] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:11.702722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded53e2-2b85-447b-b482-6627dda1272f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:11.715915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded53e2-2b85-447b-b482-6627dda1272f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:11.724482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:11.725233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:11.725659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:11.726037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:11.726820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:14:11.727259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:11.729394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:11.732015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:11.745830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 941/1882] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:14:11.750778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095191e8-c5f7-4ad5-945d-d41d29f501df None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:11.765710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095191e8-c5f7-4ad5-945d-d41d29f501df None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:11.773732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670b13b-d11b-4e16-a7c0-5b069283f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:11.774306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670b13b-d11b-4e16-a7c0-5b069283f047 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:11.774601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670b13b-d11b-4e16-a7c0-5b069283f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:11.774877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6670b13b-d11b-4e16-a7c0-5b069283f047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:11.775547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670b13b-d11b-4e16-a7c0-5b069283f047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:11.777491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6670b13b-d11b-4e16-a7c0-5b069283f047 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:11.815148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 942/1883] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:14:11 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:14:11.819392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:11.830934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716a311-739f-4bdf-82b8-86299ba4bdfb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:11.906104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8a140-c338-48d5-aaed-9ac43dcc05f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:11.907164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8a140-c338-48d5-aaed-9ac43dcc05f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:11.907579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8a140-c338-48d5-aaed-9ac43dcc05f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:11.907990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8a140-c338-48d5-aaed-9ac43dcc05f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:11.908654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8a140-c338-48d5-aaed-9ac43dcc05f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:11.910876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8a140-c338-48d5-aaed-9ac43dcc05f5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:11.941489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 942/1884] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:14:11 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:11.945545 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670b13b-d11b-4e16-a7c0-5b069283f047 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:11.956196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6670b13b-d11b-4e16-a7c0-5b069283f047 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:11.961122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be92fb2-4a0f-4234-9167-c9c14a49b292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:11.961743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be92fb2-4a0f-4234-9167-c9c14a49b292 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:11.962031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be92fb2-4a0f-4234-9167-c9c14a49b292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:11.962308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be92fb2-4a0f-4234-9167-c9c14a49b292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:11.963130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be92fb2-4a0f-4234-9167-c9c14a49b292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:11.965103 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be92fb2-4a0f-4234-9167-c9c14a49b292 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:11.976211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 943/1885] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:11.980275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8a140-c338-48d5-aaed-9ac43dcc05f5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:11.991486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8a140-c338-48d5-aaed-9ac43dcc05f5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:11.996325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475a53c-17ae-44c0-9a21-f458df093211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:11.997578 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a53c-17ae-44c0-9a21-f458df093211 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:11.997905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a53c-17ae-44c0-9a21-f458df093211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:11.998233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a53c-17ae-44c0-9a21-f458df093211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:11.998875 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475a53c-17ae-44c0-9a21-f458df093211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:12.000740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475a53c-17ae-44c0-9a21-f458df093211 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:12.012403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 943/1886] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:12.016859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be92fb2-4a0f-4234-9167-c9c14a49b292 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:12.028770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be92fb2-4a0f-4234-9167-c9c14a49b292 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:12.035138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea9ce3-4264-433d-9113-9d342660be4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:12.035346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea9ce3-4264-433d-9113-9d342660be4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.035630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea9ce3-4264-433d-9113-9d342660be4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.035911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea9ce3-4264-433d-9113-9d342660be4e admin admin] PATH_INFO: `/v3/projects/b6001fcb895a43fda2fb8876e069a2db/users/2cb12315cdc44472bd06510594564364/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.057845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea9ce3-4264-433d-9113-9d342660be4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6001fcb895a43fda2fb8876e069a2db, user_id=2cb12315cdc44472bd06510594564364, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:12.060675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea9ce3-4264-433d-9113-9d342660be4e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:12.080841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 944/1887] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:14:12 2022] PUT /identity/v3/projects/b6001fcb895a43fda2fb8876e069a2db/users/2cb12315cdc44472bd06510594564364/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:14:12.092048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b94ed-4b11-4192-a0a1-2369f13df762 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.092320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b94ed-4b11-4192-a0a1-2369f13df762 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.092598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b94ed-4b11-4192-a0a1-2369f13df762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.122648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-926b94ed-4b11-4192-a0a1-2369f13df762 None None] MFA Rules not processed for user `2cb12315cdc44472bd06510594564364`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:12.124069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-926b94ed-4b11-4192-a0a1-2369f13df762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:12.198761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 944/1888] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:14:12.203297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea9ce3-4264-433d-9113-9d342660be4e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:12.272525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea9ce3-4264-433d-9113-9d342660be4e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:12.277184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff1b57-087b-441d-bad0-d0fef4794446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:12.278297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff1b57-087b-441d-bad0-d0fef4794446 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.279105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff1b57-087b-441d-bad0-d0fef4794446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.279351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff1b57-087b-441d-bad0-d0fef4794446 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.280000 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff1b57-087b-441d-bad0-d0fef4794446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:12.281900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff1b57-087b-441d-bad0-d0fef4794446 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:12.315176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 945/1889] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:14:12 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:12.316374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b94ed-4b11-4192-a0a1-2369f13df762 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:12.330067 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b94ed-4b11-4192-a0a1-2369f13df762 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:12.421181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f4773-d275-4028-b02e-d6d1ee41abe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:12.421832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f4773-d275-4028-b02e-d6d1ee41abe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.422117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f4773-d275-4028-b02e-d6d1ee41abe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.422526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f4773-d275-4028-b02e-d6d1ee41abe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.423122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f4773-d275-4028-b02e-d6d1ee41abe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:12.425165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f4773-d275-4028-b02e-d6d1ee41abe2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:12.439336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:12.460423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 945/1890] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:14:12 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:12.466516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff1b57-087b-441d-bad0-d0fef4794446 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:12.476832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff1b57-087b-441d-bad0-d0fef4794446 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:12.481760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658311f3-887c-405a-9c83-866ad9c2427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:12.482385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658311f3-887c-405a-9c83-866ad9c2427c admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.482664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658311f3-887c-405a-9c83-866ad9c2427c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.482942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658311f3-887c-405a-9c83-866ad9c2427c admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.483638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658311f3-887c-405a-9c83-866ad9c2427c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:12.489546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658311f3-887c-405a-9c83-866ad9c2427c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:12.500435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:12.511692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 946/1891] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:12.516622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f4773-d275-4028-b02e-d6d1ee41abe2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:12.527030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f4773-d275-4028-b02e-d6d1ee41abe2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:12.535310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b21ace-3763-41a9-a2ef-4678c56fffa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:12.536961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b21ace-3763-41a9-a2ef-4678c56fffa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.537256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b21ace-3763-41a9-a2ef-4678c56fffa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.537532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b21ace-3763-41a9-a2ef-4678c56fffa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.538653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b21ace-3763-41a9-a2ef-4678c56fffa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:12.541694 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b21ace-3763-41a9-a2ef-4678c56fffa6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:12.555799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 946/1892] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:14:12.562017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658311f3-887c-405a-9c83-866ad9c2427c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:12.575403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658311f3-887c-405a-9c83-866ad9c2427c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:12.584867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700e43e-6732-47af-a6cc-9038b90ef148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:12.585731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700e43e-6732-47af-a6cc-9038b90ef148 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.586179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700e43e-6732-47af-a6cc-9038b90ef148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.586463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700e43e-6732-47af-a6cc-9038b90ef148 admin admin] PATH_INFO: `/v3/projects/01f13173bf984c85abd93e8807436185/users/1cedafe225f642318d0f67a247dc96bf/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.604252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700e43e-6732-47af-a6cc-9038b90ef148 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01f13173bf984c85abd93e8807436185, user_id=1cedafe225f642318d0f67a247dc96bf, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:12.607486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700e43e-6732-47af-a6cc-9038b90ef148 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:12.632169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 947/1893] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:14:12 2022] PUT /identity/v3/projects/01f13173bf984c85abd93e8807436185/users/1cedafe225f642318d0f67a247dc96bf/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:12.644829 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582479-e71a-46f5-a68a-e5a67ee46793 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.645153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582479-e71a-46f5-a68a-e5a67ee46793 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.645516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f582479-e71a-46f5-a68a-e5a67ee46793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.682441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0f582479-e71a-46f5-a68a-e5a67ee46793 None None] MFA Rules not processed for user `1cedafe225f642318d0f67a247dc96bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:12.683891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0f582479-e71a-46f5-a68a-e5a67ee46793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:12.772464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 947/1894] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:12.777209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700e43e-6732-47af-a6cc-9038b90ef148 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:12.875455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700e43e-6732-47af-a6cc-9038b90ef148 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:12.880953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ed2b1-db2f-4eb4-9afc-6445b4d7565e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:12.881587 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ed2b1-db2f-4eb4-9afc-6445b4d7565e admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:12.881877 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ed2b1-db2f-4eb4-9afc-6445b4d7565e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:12.882158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ed2b1-db2f-4eb4-9afc-6445b4d7565e admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:12.882804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ed2b1-db2f-4eb4-9afc-6445b4d7565e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:12.885927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9ed2b1-db2f-4eb4-9afc-6445b4d7565e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:12.923245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 948/1895] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:14:12 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:12.927025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f582479-e71a-46f5-a68a-e5a67ee46793 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:12.945671 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f582479-e71a-46f5-a68a-e5a67ee46793 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:13.019981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934e0ae-fa0d-4c56-8115-2d2d05cc437e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:13.020784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e0ae-fa0d-4c56-8115-2d2d05cc437e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:13.021641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e0ae-fa0d-4c56-8115-2d2d05cc437e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:13.022025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934e0ae-fa0d-4c56-8115-2d2d05cc437e admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:13.022841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934e0ae-fa0d-4c56-8115-2d2d05cc437e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:13.025027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934e0ae-fa0d-4c56-8115-2d2d05cc437e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:13.062373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 948/1896] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:14:12 2022] POST /identity/v3/users => generated 298 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:13.070026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ed2b1-db2f-4eb4-9afc-6445b4d7565e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:13.082295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ed2b1-db2f-4eb4-9afc-6445b4d7565e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:13.087271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33bea-8df9-4f24-9b4d-4afbf9de8068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:13.087892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33bea-8df9-4f24-9b4d-4afbf9de8068 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:13.088177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33bea-8df9-4f24-9b4d-4afbf9de8068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:13.088456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33bea-8df9-4f24-9b4d-4afbf9de8068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:13.089121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead33bea-8df9-4f24-9b4d-4afbf9de8068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:13.091306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead33bea-8df9-4f24-9b4d-4afbf9de8068 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:13.103194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 949/1897] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:13.108310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934e0ae-fa0d-4c56-8115-2d2d05cc437e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:13.119418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934e0ae-fa0d-4c56-8115-2d2d05cc437e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:13.124579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25a719-48ff-45ca-81a1-ed2afcc56cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:13.125244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25a719-48ff-45ca-81a1-ed2afcc56cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:13.125528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25a719-48ff-45ca-81a1-ed2afcc56cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:13.125929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25a719-48ff-45ca-81a1-ed2afcc56cf4 admin admin] PATH_INFO: `/v3/projects/016d211a0c8d4da6b3e26b794fe5c4a2/users/3a623fdf56eb4de68f476fce6a55bc9e/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:13.143752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25a719-48ff-45ca-81a1-ed2afcc56cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016d211a0c8d4da6b3e26b794fe5c4a2, user_id=3a623fdf56eb4de68f476fce6a55bc9e, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:13.146971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d25a719-48ff-45ca-81a1-ed2afcc56cf4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:13.165765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 949/1898] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:14:13 2022] PUT /identity/v3/projects/016d211a0c8d4da6b3e26b794fe5c4a2/users/3a623fdf56eb4de68f476fce6a55bc9e/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:14:13.170425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33bea-8df9-4f24-9b4d-4afbf9de8068 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:13.181050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33bea-8df9-4f24-9b4d-4afbf9de8068 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:13.247673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fee9a4-224d-4544-b7e5-53e24aca106b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:13.248289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee9a4-224d-4544-b7e5-53e24aca106b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:13.248576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee9a4-224d-4544-b7e5-53e24aca106b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:13.248856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fee9a4-224d-4544-b7e5-53e24aca106b admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:13.249504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fee9a4-224d-4544-b7e5-53e24aca106b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:13.251143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fee9a4-224d-4544-b7e5-53e24aca106b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:13.261630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 950/1899] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:14:13.266458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25a719-48ff-45ca-81a1-ed2afcc56cf4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:13.276871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d25a719-48ff-45ca-81a1-ed2afcc56cf4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:13.281656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:13.282309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:13.282595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:13.282874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] PATH_INFO: `/v3/projects/016d211a0c8d4da6b3e26b794fe5c4a2/users/3a623fdf56eb4de68f476fce6a55bc9e/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:13.284810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016d211a0c8d4da6b3e26b794fe5c4a2, user_id=3a623fdf56eb4de68f476fce6a55bc9e, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:13.287601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:13.298150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 950/1900] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:14:13 2022] PUT /identity/v3/projects/016d211a0c8d4da6b3e26b794fe5c4a2/users/3a623fdf56eb4de68f476fce6a55bc9e/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:13.308749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b30a4-1256-4f75-869b-685b93d55b53 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:13.309605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b30a4-1256-4f75-869b-685b93d55b53 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:13.309987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b30a4-1256-4f75-869b-685b93d55b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:13.338912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-034b30a4-1256-4f75-869b-685b93d55b53 None None] MFA Rules not processed for user `3a623fdf56eb4de68f476fce6a55bc9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:13.340261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-034b30a4-1256-4f75-869b-685b93d55b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:13.375177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:13.413752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 951/1901] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:13.426306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:13.429187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:13.459244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:13.491978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c510ee4-1c4a-4592-99c4-363e176a4a70 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:13.495654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb84d3-2501-4bca-bf6a-d96a8de78209 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:13.496263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb84d3-2501-4bca-bf6a-d96a8de78209 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:13.496615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb84d3-2501-4bca-bf6a-d96a8de78209 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:13.497399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb84d3-2501-4bca-bf6a-d96a8de78209 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:13.554343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb84d3-2501-4bca-bf6a-d96a8de78209 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:13.555724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb84d3-2501-4bca-bf6a-d96a8de78209 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:13.565642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 951/1902] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:15.083193 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b30a4-1256-4f75-869b-685b93d55b53 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:15.093391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b30a4-1256-4f75-869b-685b93d55b53 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:15.097194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09315835-5cd3-4f27-bde3-a0570eb56e3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:15.097825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09315835-5cd3-4f27-bde3-a0570eb56e3d service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:15.098105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09315835-5cd3-4f27-bde3-a0570eb56e3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:15.098376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09315835-5cd3-4f27-bde3-a0570eb56e3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:15.156554 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09315835-5cd3-4f27-bde3-a0570eb56e3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:15.157978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09315835-5cd3-4f27-bde3-a0570eb56e3d service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:15.166706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 952/1903] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:15.975342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb84d3-2501-4bca-bf6a-d96a8de78209 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:15.983843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb84d3-2501-4bca-bf6a-d96a8de78209 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:15.987514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81847c84-8188-4ff5-ba83-a7a0459e030c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:15.988137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81847c84-8188-4ff5-ba83-a7a0459e030c service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:15.988419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81847c84-8188-4ff5-ba83-a7a0459e030c service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:15.988723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81847c84-8188-4ff5-ba83-a7a0459e030c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:15.998248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81847c84-8188-4ff5-ba83-a7a0459e030c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:15.999723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81847c84-8188-4ff5-ba83-a7a0459e030c service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:16.009354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 952/1904] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:15 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:14:20.950461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09315835-5cd3-4f27-bde3-a0570eb56e3d service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:21.012249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09315835-5cd3-4f27-bde3-a0570eb56e3d service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:21.016738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:21.017363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:21.017646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:21.017923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] PATH_INFO: `/v3/users/2cb12315cdc44472bd06510594564364` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:21.019033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb12315cdc44472bd06510594564364)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:21.021020 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:21.067489 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb12315cdc44472bd06510594564364'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.079977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb12315cdc44472bd06510594564364'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.080698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb12315cdc44472bd06510594564364'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.088862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb12315cdc44472bd06510594564364'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.090074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:21.098776 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb12315cdc44472bd06510594564364'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.104736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 953/1905] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:20 2022] DELETE /identity/v3/users/2cb12315cdc44472bd06510594564364 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:21.109802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528a0af-5b75-431b-aff9-8f7aa8b9a560 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:21.110272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528a0af-5b75-431b-aff9-8f7aa8b9a560 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:21.110457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d528a0af-5b75-431b-aff9-8f7aa8b9a560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:21.130625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d528a0af-5b75-431b-aff9-8f7aa8b9a560 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:21.131939 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d528a0af-5b75-431b-aff9-8f7aa8b9a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:21.201115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 953/1906] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:21.210709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:21.212264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:21.272167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c413508-9730-48a8-89aa-ef94f07d54ce admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:21.275962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f3e00-0893-426e-b99a-a4127d97338f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:21.276570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3e00-0893-426e-b99a-a4127d97338f service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:21.276852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3e00-0893-426e-b99a-a4127d97338f service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:21.277128 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f3e00-0893-426e-b99a-a4127d97338f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:21.287727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f3e00-0893-426e-b99a-a4127d97338f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:21.289174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f3e00-0893-426e-b99a-a4127d97338f service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:21.299537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 954/1907] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:14:21.714669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528a0af-5b75-431b-aff9-8f7aa8b9a560 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:21.716322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d528a0af-5b75-431b-aff9-8f7aa8b9a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:21.740000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d528a0af-5b75-431b-aff9-8f7aa8b9a560 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:21.746155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:21.747408 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:21.747696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:21.747973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] PATH_INFO: `/v3/projects/b6001fcb895a43fda2fb8876e069a2db` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:21.749120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6001fcb895a43fda2fb8876e069a2db)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:21.751098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:21.804397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6001fcb895a43fda2fb8876e069a2db', 'request_id': 'req-4f461422-8443-4e1a-8071-5253193947bd'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.811907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6001fcb895a43fda2fb8876e069a2db', 'request_id': 'req-4f461422-8443-4e1a-8071-5253193947bd'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.817194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:21.821735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6001fcb895a43fda2fb8876e069a2db', 'request_id': 'req-4f461422-8443-4e1a-8071-5253193947bd'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.837963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] The token cache is being invalidate because project b6001fcb895a43fda2fb8876e069a2db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:21.837963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.837963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.839590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 954/1908] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:21 2022] DELETE /identity/v3/projects/b6001fcb895a43fda2fb8876e069a2db => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:21.842759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f3e00-0893-426e-b99a-a4127d97338f service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:21.852438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f3e00-0893-426e-b99a-a4127d97338f service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:21.915371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:21.916173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:21.916473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:21.916723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] PATH_INFO: `/v3/users/1cedafe225f642318d0f67a247dc96bf` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:21.917924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cedafe225f642318d0f67a247dc96bf)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:21.920320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:21.978774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cedafe225f642318d0f67a247dc96bf'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.993036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cedafe225f642318d0f67a247dc96bf'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:21.993755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cedafe225f642318d0f67a247dc96bf'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.001798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cedafe225f642318d0f67a247dc96bf'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.008819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cedafe225f642318d0f67a247dc96bf'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.015480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 955/1909] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:21 2022] DELETE /identity/v3/users/1cedafe225f642318d0f67a247dc96bf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:22.343579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:22.345013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:22.360870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f461422-8443-4e1a-8071-5253193947bd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:22.420544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:22.421114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:22.421397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:22.421673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] PATH_INFO: `/v3/projects/01f13173bf984c85abd93e8807436185` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:22.422797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01f13173bf984c85abd93e8807436185)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:22.424743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:22.472131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01f13173bf984c85abd93e8807436185', 'request_id': 'req-bc180e23-b15b-420c-9d6a-46681ab46067'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.478416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01f13173bf984c85abd93e8807436185', 'request_id': 'req-bc180e23-b15b-420c-9d6a-46681ab46067'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.487802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01f13173bf984c85abd93e8807436185', 'request_id': 'req-bc180e23-b15b-420c-9d6a-46681ab46067'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.499363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] The token cache is being invalidate because project 01f13173bf984c85abd93e8807436185 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:22.499653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.500368 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.502005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 955/1910] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:22 2022] DELETE /identity/v3/projects/01f13173bf984c85abd93e8807436185 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:22.506356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:22.507775 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:22.521266 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7b4cd-7e02-43e9-af5f-9a35678f7f16 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:22.529945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:22.595177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:22.596460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:22.596747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:22.597034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] PATH_INFO: `/v3/users/3a623fdf56eb4de68f476fce6a55bc9e` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:22.598191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a623fdf56eb4de68f476fce6a55bc9e)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:22.600435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:22.667943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a623fdf56eb4de68f476fce6a55bc9e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.685917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a623fdf56eb4de68f476fce6a55bc9e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.687042 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a623fdf56eb4de68f476fce6a55bc9e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.696152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a623fdf56eb4de68f476fce6a55bc9e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.709442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f664b505-55c1-4d48-8a21-11e4520589a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a623fdf56eb4de68f476fce6a55bc9e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:22.719859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 956/1911] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:22 2022] DELETE /identity/v3/users/3a623fdf56eb4de68f476fce6a55bc9e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:23.370791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:23.372839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:23.403116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc180e23-b15b-420c-9d6a-46681ab46067 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:23.475184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:23.475825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.476289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.476557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] PATH_INFO: `/v3/projects/016d211a0c8d4da6b3e26b794fe5c4a2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.477678 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016d211a0c8d4da6b3e26b794fe5c4a2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:23.479665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:23.502677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:23.531496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016d211a0c8d4da6b3e26b794fe5c4a2', 'request_id': 'req-58a782e8-cb9d-4b56-b115-e752016eeb94'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.537919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016d211a0c8d4da6b3e26b794fe5c4a2', 'request_id': 'req-58a782e8-cb9d-4b56-b115-e752016eeb94'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.545750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016d211a0c8d4da6b3e26b794fe5c4a2', 'request_id': 'req-58a782e8-cb9d-4b56-b115-e752016eeb94'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.556822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] The token cache is being invalidate because project 016d211a0c8d4da6b3e26b794fe5c4a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:23.557115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.557852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-58a782e8-cb9d-4b56-b115-e752016eeb94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.560569 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 956/1912] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:23 2022] DELETE /identity/v3/projects/016d211a0c8d4da6b3e26b794fe5c4a2 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:23.566041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e60109-1015-4868-a417-3bd753285cf6 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.566553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e60109-1015-4868-a417-3bd753285cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.566642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e60109-1015-4868-a417-3bd753285cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.587788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-85e60109-1015-4868-a417-3bd753285cf6 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:23.592267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-85e60109-1015-4868-a417-3bd753285cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:23.654771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 957/1913] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:23.663649 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66d37-13ac-4634-87cb-6fd2a53af880 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.663942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66d37-13ac-4634-87cb-6fd2a53af880 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.664227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66d37-13ac-4634-87cb-6fd2a53af880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.686073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-acf66d37-13ac-4634-87cb-6fd2a53af880 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:23.687310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-acf66d37-13ac-4634-87cb-6fd2a53af880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:23.695944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 957/1914] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:14:23.700781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e60109-1015-4868-a417-3bd753285cf6 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:23.710297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e60109-1015-4868-a417-3bd753285cf6 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:23.714959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:23.715703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.716052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.716395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.717111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:14:23.717424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:23.719022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:23.730819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 958/1915] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:14:23.735676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf66d37-13ac-4634-87cb-6fd2a53af880 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:23.746795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf66d37-13ac-4634-87cb-6fd2a53af880 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:23.753315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967b022-3d1a-4a09-9897-43e8ac6dfdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:23.753970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967b022-3d1a-4a09-9897-43e8ac6dfdae admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.754256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967b022-3d1a-4a09-9897-43e8ac6dfdae admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.754536 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967b022-3d1a-4a09-9897-43e8ac6dfdae admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.755215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967b022-3d1a-4a09-9897-43e8ac6dfdae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:23.757236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967b022-3d1a-4a09-9897-43e8ac6dfdae admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:23.766679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:23.769017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:23.784350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa351da-8aed-4bb5-a54b-a1477ef9d5da admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:23.786846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 958/1916] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:14:23 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:23.789441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:23.790078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.790365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.790642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] PATH_INFO: `/v3/users/3326a853fc3a43c283f5750fb9fc650b` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.791127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967b022-3d1a-4a09-9897-43e8ac6dfdae admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:23.791983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3326a853fc3a43c283f5750fb9fc650b)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:23.794316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:23.807937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:23.846940 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3326a853fc3a43c283f5750fb9fc650b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.858677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967b022-3d1a-4a09-9897-43e8ac6dfdae admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:23.860042 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3326a853fc3a43c283f5750fb9fc650b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.860751 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3326a853fc3a43c283f5750fb9fc650b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.863331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:23.863936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.864215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.864489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.865121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:23.867033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:23.871990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3326a853fc3a43c283f5750fb9fc650b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.880799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a53f8576-1819-4147-bd5d-93268bc04e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3326a853fc3a43c283f5750fb9fc650b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:23.888800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 959/1917] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:23 2022] DELETE /identity/v3/users/3326a853fc3a43c283f5750fb9fc650b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:14:23.896221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 959/1918] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:14:23 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:23.897100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4673c-407b-410f-b968-2d082b613b23 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.897302 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:23.897404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4673c-407b-410f-b968-2d082b613b23 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.897711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4673c-407b-410f-b968-2d082b613b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.899076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:23.911846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f103d9d-e9da-43e9-9ec8-2c7f3fa98982 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:23.917134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c3f4673c-407b-410f-b968-2d082b613b23 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:23.918881 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c3f4673c-407b-410f-b968-2d082b613b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:23.978572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ed43a-ee70-4818-bceb-a6d3c1b34adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:23.979227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ed43a-ee70-4818-bceb-a6d3c1b34adb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:23.979511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ed43a-ee70-4818-bceb-a6d3c1b34adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:23.979786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ed43a-ee70-4818-bceb-a6d3c1b34adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:23.980430 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ed43a-ee70-4818-bceb-a6d3c1b34adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:23.982785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ed43a-ee70-4818-bceb-a6d3c1b34adb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:23.986767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 960/1919] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:14:23.997232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4673c-407b-410f-b968-2d082b613b23 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:23.997395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 960/1920] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:14:23.998869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c3f4673c-407b-410f-b968-2d082b613b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:24.001487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ed43a-ee70-4818-bceb-a6d3c1b34adb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.007793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:24.012652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ed43a-ee70-4818-bceb-a6d3c1b34adb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.019293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab691a-9a9b-4881-a9cb-61b0a9c3f2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.019987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab691a-9a9b-4881-a9cb-61b0a9c3f2aa admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.020778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab691a-9a9b-4881-a9cb-61b0a9c3f2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.020778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab691a-9a9b-4881-a9cb-61b0a9c3f2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.021481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab691a-9a9b-4881-a9cb-61b0a9c3f2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.024339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab691a-9a9b-4881-a9cb-61b0a9c3f2aa admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.036749 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 961/1921] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:14:24.040575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab691a-9a9b-4881-a9cb-61b0a9c3f2aa admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.052278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab691a-9a9b-4881-a9cb-61b0a9c3f2aa admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.057668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.058328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.058760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.060195 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] PATH_INFO: `/v3/projects/7c665b0a9fe64be4b160285e344ee4e6/users/9deac6d3ef204b66a47b0e4e83fab856/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.063413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4673c-407b-410f-b968-2d082b613b23 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.067409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2cd686-77a1-4bd2-8070-594e16ebaded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.068037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cd686-77a1-4bd2-8070-594e16ebaded service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.068322 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cd686-77a1-4bd2-8070-594e16ebaded service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.068601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2cd686-77a1-4bd2-8070-594e16ebaded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.079297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2cd686-77a1-4bd2-8070-594e16ebaded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.080658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2cd686-77a1-4bd2-8070-594e16ebaded service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.089812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c665b0a9fe64be4b160285e344ee4e6, user_id=9deac6d3ef204b66a47b0e4e83fab856, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.090502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 961/1922] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:14:24.092712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.107047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 962/1923] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:14:24 2022] PUT /identity/v3/projects/7c665b0a9fe64be4b160285e344ee4e6/users/9deac6d3ef204b66a47b0e4e83fab856/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:14:24.117539 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5b12b-73ca-4f22-ba29-a90e95911b08 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.118225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5b12b-73ca-4f22-ba29-a90e95911b08 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.118520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5b12b-73ca-4f22-ba29-a90e95911b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.148681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-33f5b12b-73ca-4f22-ba29-a90e95911b08 None None] MFA Rules not processed for user `9deac6d3ef204b66a47b0e4e83fab856`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:24.149901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-33f5b12b-73ca-4f22-ba29-a90e95911b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:24.172299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:24.222038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 962/1924] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:24.234193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.235925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:24.298627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199233fa-33e9-4594-bfe5-9fcac9f36938 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.302262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.302870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.303188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.303464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.314197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.316569 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.327093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 963/1925] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:24.354920 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5b12b-73ca-4f22-ba29-a90e95911b08 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.375973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.377635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:24.438232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5b12b-73ca-4f22-ba29-a90e95911b08 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.443135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.443885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.444264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.444639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] PATH_INFO: `/v3/users/9deac6d3ef204b66a47b0e4e83fab856` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.445879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9deac6d3ef204b66a47b0e4e83fab856)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.447952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.465596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86335c-4f86-45d8-a9c0-a3ad4056a281 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.470396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.471779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.472066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.472345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] PATH_INFO: `/v3/projects/57b1e0cfce26405d9ae4182e7acd9a21` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.473488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b1e0cfce26405d9ae4182e7acd9a21)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.475554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.505888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9deac6d3ef204b66a47b0e4e83fab856'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.522811 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9deac6d3ef204b66a47b0e4e83fab856'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.523800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9deac6d3ef204b66a47b0e4e83fab856'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.531358 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b1e0cfce26405d9ae4182e7acd9a21', 'request_id': 'req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.532758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9deac6d3ef204b66a47b0e4e83fab856'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.539041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b1e0cfce26405d9ae4182e7acd9a21', 'request_id': 'req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.542258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0255f714-0179-4252-85d1-003b05498138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9deac6d3ef204b66a47b0e4e83fab856'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.548253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b1e0cfce26405d9ae4182e7acd9a21', 'request_id': 'req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.551280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 963/1926] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:14:24 2022] DELETE /identity/v3/users/9deac6d3ef204b66a47b0e4e83fab856 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:24.556471 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28874-d9c6-4910-98fd-f1c36b7ac8f8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.557140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28874-d9c6-4910-98fd-f1c36b7ac8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.557975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb28874-d9c6-4910-98fd-f1c36b7ac8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.562597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] The token cache is being invalidate because project 57b1e0cfce26405d9ae4182e7acd9a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:24.562985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.563818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8f22b7c7-9d2f-4863-ad90-e68e68cdffd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:24.565613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 964/1927] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:24 2022] DELETE /identity/v3/projects/57b1e0cfce26405d9ae4182e7acd9a21 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:14:24.570443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f7ff-4e1b-478d-9659-a00b36d4a40a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.570732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f7ff-4e1b-478d-9659-a00b36d4a40a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.571016 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26f7ff-4e1b-478d-9659-a00b36d4a40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.580249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6eb28874-d9c6-4910-98fd-f1c36b7ac8f8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:24.581469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6eb28874-d9c6-4910-98fd-f1c36b7ac8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:24.582502 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:24.597491 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3d26f7ff-4e1b-478d-9659-a00b36d4a40a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:24.598535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3d26f7ff-4e1b-478d-9659-a00b36d4a40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:24.625257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:24.650384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 964/1928] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:14:24.660154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb28874-d9c6-4910-98fd-f1c36b7ac8f8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.661557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6eb28874-d9c6-4910-98fd-f1c36b7ac8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:24.680734 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 965/1929] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:24.689560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f74e-9d82-4cbe-bcc7-2b58c59326a2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.689849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f74e-9d82-4cbe-bcc7-2b58c59326a2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.690127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f74e-9d82-4cbe-bcc7-2b58c59326a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.709390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7926f74e-9d82-4cbe-bcc7-2b58c59326a2 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:24.710596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7926f74e-9d82-4cbe-bcc7-2b58c59326a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:24.718658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 966/1930] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:14:24.722963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926f74e-9d82-4cbe-bcc7-2b58c59326a2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.723981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb28874-d9c6-4910-98fd-f1c36b7ac8f8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.729897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1722158-55eb-4ec2-b14c-a39f5bf2f6bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.730460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1722158-55eb-4ec2-b14c-a39f5bf2f6bf service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.731307 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1722158-55eb-4ec2-b14c-a39f5bf2f6bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.731793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1722158-55eb-4ec2-b14c-a39f5bf2f6bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.732762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926f74e-9d82-4cbe-bcc7-2b58c59326a2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.738591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.739356 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.739570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.739843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.740542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:14:24.740872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.742495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.749095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1722158-55eb-4ec2-b14c-a39f5bf2f6bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.750663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1722158-55eb-4ec2-b14c-a39f5bf2f6bf service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.756082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 967/1931] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:14:24.760918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.762214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 965/1932] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:14:24.772191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89b1a5-2fb3-4dee-a268-c005c7279928 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.777283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ccf62-f313-4aec-9ac3-7d3cc4660ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.778051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ccf62-f313-4aec-9ac3-7d3cc4660ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.778401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ccf62-f313-4aec-9ac3-7d3cc4660ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.778769 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ccf62-f313-4aec-9ac3-7d3cc4660ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.779758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ccf62-f313-4aec-9ac3-7d3cc4660ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.781790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ccf62-f313-4aec-9ac3-7d3cc4660ae4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.811093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 968/1933] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:14:24 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:14:24.815974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1722158-55eb-4ec2-b14c-a39f5bf2f6bf service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.827640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1722158-55eb-4ec2-b14c-a39f5bf2f6bf service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.902036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecf2dd-2aca-4d2f-a566-eb70d6025e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.902898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecf2dd-2aca-4d2f-a566-eb70d6025e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.903248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecf2dd-2aca-4d2f-a566-eb70d6025e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.903640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecf2dd-2aca-4d2f-a566-eb70d6025e2e admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.904962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecf2dd-2aca-4d2f-a566-eb70d6025e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.910723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecf2dd-2aca-4d2f-a566-eb70d6025e2e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.942443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 966/1934] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:14:24 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:14:24.943714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ccf62-f313-4aec-9ac3-7d3cc4660ae4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.954299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ccf62-f313-4aec-9ac3-7d3cc4660ae4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.958996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf614c-0c8f-4993-afdb-3f3e493f7249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.959802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf614c-0c8f-4993-afdb-3f3e493f7249 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.960089 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf614c-0c8f-4993-afdb-3f3e493f7249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.960362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf614c-0c8f-4993-afdb-3f3e493f7249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:24.961006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf614c-0c8f-4993-afdb-3f3e493f7249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:24.962605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf614c-0c8f-4993-afdb-3f3e493f7249 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:24.975479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 969/1935] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:14:24.976542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecf2dd-2aca-4d2f-a566-eb70d6025e2e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:24.991603 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecf2dd-2aca-4d2f-a566-eb70d6025e2e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:24.997750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ea1eb0-ec71-458c-a3fa-bd7d4d14f260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:24.998888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea1eb0-ec71-458c-a3fa-bd7d4d14f260 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:24.999279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea1eb0-ec71-458c-a3fa-bd7d4d14f260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:24.999621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ea1eb0-ec71-458c-a3fa-bd7d4d14f260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:25.000655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea1eb0-ec71-458c-a3fa-bd7d4d14f260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:25.002458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ea1eb0-ec71-458c-a3fa-bd7d4d14f260 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:25.013681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 967/1936] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:14:25.017590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf614c-0c8f-4993-afdb-3f3e493f7249 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:25.031359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf614c-0c8f-4993-afdb-3f3e493f7249 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:25.036019 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:25.036680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:25.036964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:25.037239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] PATH_INFO: `/v3/projects/f027aba5a38b422fbc30d64b9ee2ef79/users/abd7bde497c14d25a06c276609aa143f/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:25.052578 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f027aba5a38b422fbc30d64b9ee2ef79, user_id=abd7bde497c14d25a06c276609aa143f, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:25.055318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:25.071346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 970/1937] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:14:25 2022] PUT /identity/v3/projects/f027aba5a38b422fbc30d64b9ee2ef79/users/abd7bde497c14d25a06c276609aa143f/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:14:25.076463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9215f-1238-4d9b-98a1-ed9b085e0e9c None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:25.076870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9215f-1238-4d9b-98a1-ed9b085e0e9c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:25.077279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b9215f-1238-4d9b-98a1-ed9b085e0e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:25.085197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:25.088176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:25.130083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-14b9215f-1238-4d9b-98a1-ed9b085e0e9c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:25.131864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-14b9215f-1238-4d9b-98a1-ed9b085e0e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:25.158556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a3860-8435-4093-b860-456feca510f3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:25.163780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:25.164461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:25.164816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:25.165163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] PATH_INFO: `/v3/projects/7c665b0a9fe64be4b160285e344ee4e6` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:25.166393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c665b0a9fe64be4b160285e344ee4e6)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:25.169501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:25.203759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 968/1938] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:25.205426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:25.215379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9215f-1238-4d9b-98a1-ed9b085e0e9c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:25.217496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-14b9215f-1238-4d9b-98a1-ed9b085e0e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:25.224386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:25.235691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c665b0a9fe64be4b160285e344ee4e6', 'request_id': 'req-25134033-db13-42bb-9980-d397cf5744bf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:25.242742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c665b0a9fe64be4b160285e344ee4e6', 'request_id': 'req-25134033-db13-42bb-9980-d397cf5744bf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:25.251768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c665b0a9fe64be4b160285e344ee4e6', 'request_id': 'req-25134033-db13-42bb-9980-d397cf5744bf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:25.264417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] The token cache is being invalidate because project 7c665b0a9fe64be4b160285e344ee4e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:14:25.264867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:25.265643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25134033-db13-42bb-9980-d397cf5744bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:14:25.268114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 971/1939] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:14:25 2022] DELETE /identity/v3/projects/7c665b0a9fe64be4b160285e344ee4e6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:14:25.294882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b9215f-1238-4d9b-98a1-ed9b085e0e9c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:25.356945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:25.357611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:25.357886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:25.358152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:25.359942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:25.429184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:25.430804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:25.443339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 969/1940] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:26.224486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce555c-d6a5-4ca4-bf38-c866e7c40e0b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:26.224980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce555c-d6a5-4ca4-bf38-c866e7c40e0b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:26.225061 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce555c-d6a5-4ca4-bf38-c866e7c40e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:26.262504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-73ce555c-d6a5-4ca4-bf38-c866e7c40e0b None None] MFA Rules not processed for user `abd7bde497c14d25a06c276609aa143f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:26.263969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-73ce555c-d6a5-4ca4-bf38-c866e7c40e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:26.335200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 972/1941] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:26.347747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:26.349763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:26.429294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98e18e-6d4c-4c5c-8684-d0009141e786 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:26.433373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce830b-7932-4388-9e7b-9d78295a1925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:26.434765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce830b-7932-4388-9e7b-9d78295a1925 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:26.435231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce830b-7932-4388-9e7b-9d78295a1925 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:26.435608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce830b-7932-4388-9e7b-9d78295a1925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:26.445433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce830b-7932-4388-9e7b-9d78295a1925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:26.447087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce830b-7932-4388-9e7b-9d78295a1925 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:26.457955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 970/1942] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:14:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:14:27.906646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c783-68db-4cdf-971d-5098787207f3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:27.907803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c783-68db-4cdf-971d-5098787207f3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:27.907803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004c783-68db-4cdf-971d-5098787207f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:27.928731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f004c783-68db-4cdf-971d-5098787207f3 None None] MFA Rules not processed for user `abd7bde497c14d25a06c276609aa143f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:27.930156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f004c783-68db-4cdf-971d-5098787207f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:27.937621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 973/1943] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:14:30.643480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e08c0-6653-4de6-8101-237e382e93d5 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:30.644402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e08c0-6653-4de6-8101-237e382e93d5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:30.644853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e08c0-6653-4de6-8101-237e382e93d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:30.667658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c85e08c0-6653-4de6-8101-237e382e93d5 None None] MFA Rules not processed for user `d29ff25b062346f6bcf9910b1fa8db5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:30.669667 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c85e08c0-6653-4de6-8101-237e382e93d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:30.681364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:30.734291 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 971/1944] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:30.908429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69611d8-9bd7-4d74-acc7-d228702b2041 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:30.908965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69611d8-9bd7-4d74-acc7-d228702b2041 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:30.908965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69611d8-9bd7-4d74-acc7-d228702b2041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:30.929427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e69611d8-9bd7-4d74-acc7-d228702b2041 None None] MFA Rules not processed for user `1fd3d61c9fad4ac8ae26433ec77cd9c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:30.930788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e69611d8-9bd7-4d74-acc7-d228702b2041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:30.993537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 974/1945] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:31.535349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e74e-bde9-4091-b95e-bd023488b87a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:31.535642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e74e-bde9-4091-b95e-bd023488b87a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:31.535926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8e74e-bde9-4091-b95e-bd023488b87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:31.559208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-afb8e74e-bde9-4091-b95e-bd023488b87a None None] MFA Rules not processed for user `c5619c8183e04e1d9073f1b40e057967`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:31.560698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-afb8e74e-bde9-4091-b95e-bd023488b87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:31.647498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 972/1946] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:14:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:40.985612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=74,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:41.634626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=44 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:43.054482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa946f-ca7c-4c8e-a679-348e9543fc3c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:43.054968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa946f-ca7c-4c8e-a679-348e9543fc3c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:43.055088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa946f-ca7c-4c8e-a679-348e9543fc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:43.090496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-adfa946f-ca7c-4c8e-a679-348e9543fc3c None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:14:43.091759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-adfa946f-ca7c-4c8e-a679-348e9543fc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:43.159668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 975/1947] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:14:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:14:43.208359 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8e74e-bde9-4091-b95e-bd023488b87a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:14:43.211286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-afb8e74e-bde9-4091-b95e-bd023488b87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:14:43.311289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8e74e-bde9-4091-b95e-bd023488b87a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:14:43.316480 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9660828-d27d-4bb0-8b57-ebe7831ef571 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:14:43.317090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9660828-d27d-4bb0-8b57-ebe7831ef571 service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:14:43.317371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9660828-d27d-4bb0-8b57-ebe7831ef571 service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:14:43.317647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9660828-d27d-4bb0-8b57-ebe7831ef571 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:14:43.325969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9660828-d27d-4bb0-8b57-ebe7831ef571 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:14:43.327573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9660828-d27d-4bb0-8b57-ebe7831ef571 service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:14:43.336683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 973/1948] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:14:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:14:53.151946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=25 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:14:53.332477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=32 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:02.674816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed59ce-254b-4a48-aab8-b903948dad00 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:02.675871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed59ce-254b-4a48-aab8-b903948dad00 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:02.676346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed59ce-254b-4a48-aab8-b903948dad00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:02.702873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d9ed59ce-254b-4a48-aab8-b903948dad00 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:02.704896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d9ed59ce-254b-4a48-aab8-b903948dad00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:02.714089 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 976/1949] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:15:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:02.735298 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9660828-d27d-4bb0-8b57-ebe7831ef571 service swift] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:02.748663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9660828-d27d-4bb0-8b57-ebe7831ef571 service swift] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:02.754917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501f4f4-dbbc-469e-9ef5-ffccec6e4884 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:02.755641 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501f4f4-dbbc-469e-9ef5-ffccec6e4884 service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:02.756140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501f4f4-dbbc-469e-9ef5-ffccec6e4884 service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:02.756822 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501f4f4-dbbc-469e-9ef5-ffccec6e4884 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:02.767786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501f4f4-dbbc-469e-9ef5-ffccec6e4884 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:02.769870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501f4f4-dbbc-469e-9ef5-ffccec6e4884 service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:02.780114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 974/1950] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:15:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:15:03.310220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b263e8e-d6b7-4a62-9a99-3da896e6f015 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.310755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b263e8e-d6b7-4a62-9a99-3da896e6f015 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.311303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b263e8e-d6b7-4a62-9a99-3da896e6f015 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:03.313078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 977/1951] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:15:03.321671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786731b-6800-408d-b9e4-febedd480131 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.322190 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786731b-6800-408d-b9e4-febedd480131 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.322465 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786731b-6800-408d-b9e4-febedd480131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:03.350644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d786731b-6800-408d-b9e4-febedd480131 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:03.353315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d786731b-6800-408d-b9e4-febedd480131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:03.480799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 975/1952] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:15:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:03.489426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b263e8e-d6b7-4a62-9a99-3da896e6f015 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:03.492183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5b263e8e-d6b7-4a62-9a99-3da896e6f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:03.573441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b263e8e-d6b7-4a62-9a99-3da896e6f015 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:03.577803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e5576-c860-4222-8014-366bba7658c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d0fc260e254d2d80c6927c52cd8a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:03.578498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e5576-c860-4222-8014-366bba7658c2 service glance] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.578904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e5576-c860-4222-8014-366bba7658c2 service glance] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.579278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e5576-c860-4222-8014-366bba7658c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:03.590947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e5576-c860-4222-8014-366bba7658c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:03.592472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e5576-c860-4222-8014-366bba7658c2 service glance] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:03.607250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 978/1953] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:15:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:15:03.694720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488a8a9-c0fc-4017-9f01-e5cd400fe92c None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.695453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488a8a9-c0fc-4017-9f01-e5cd400fe92c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.696277 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488a8a9-c0fc-4017-9f01-e5cd400fe92c None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:03.699364 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 976/1954] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:15:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:15:03.704739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2579-6dd0-4942-a5fc-cc9886cb300a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.705739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2579-6dd0-4942-a5fc-cc9886cb300a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.706287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2579-6dd0-4942-a5fc-cc9886cb300a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:03.732084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2a8b2579-6dd0-4942-a5fc-cc9886cb300a None None] MFA Rules not processed for user `9c414b04fe0544e3a938661bc957c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:03.733703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2a8b2579-6dd0-4942-a5fc-cc9886cb300a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:03.740944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 979/1955] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:15:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:15:03.755788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488a8a9-c0fc-4017-9f01-e5cd400fe92c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:03.757766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b488a8a9-c0fc-4017-9f01-e5cd400fe92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:03.777406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488a8a9-c0fc-4017-9f01-e5cd400fe92c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:03.781931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3f2bb-d208-4ce5-8baa-6d5132ca082c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:03.782578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3f2bb-d208-4ce5-8baa-6d5132ca082c service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.783030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3f2bb-d208-4ce5-8baa-6d5132ca082c service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.783340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3f2bb-d208-4ce5-8baa-6d5132ca082c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:03.792788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3f2bb-d208-4ce5-8baa-6d5132ca082c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:03.794863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3f2bb-d208-4ce5-8baa-6d5132ca082c service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:03.803870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 977/1956] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:15:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:15:03.902761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa91f3e-8fad-4496-b554-8919b52a4f47 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.903802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa91f3e-8fad-4496-b554-8919b52a4f47 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.904243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa91f3e-8fad-4496-b554-8919b52a4f47 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:03.906901 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 980/1957] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:15:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:15:03.913060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaf5a0-95d0-46bd-b327-a14e307c8c44 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.913730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaf5a0-95d0-46bd-b327-a14e307c8c44 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.914012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcaf5a0-95d0-46bd-b327-a14e307c8c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:03.936441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4bcaf5a0-95d0-46bd-b327-a14e307c8c44 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:03.937799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4bcaf5a0-95d0-46bd-b327-a14e307c8c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:03.950117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 978/1958] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:15:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:03.962154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa91f3e-8fad-4496-b554-8919b52a4f47 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:03.979625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa91f3e-8fad-4496-b554-8919b52a4f47 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:03.984349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0b9c3-57a2-4192-9378-b5bc425693a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:03.985534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b9c3-57a2-4192-9378-b5bc425693a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:03.986071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b9c3-57a2-4192-9378-b5bc425693a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:03.986588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0b9c3-57a2-4192-9378-b5bc425693a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:04.001586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0b9c3-57a2-4192-9378-b5bc425693a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:04.003435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd0b9c3-57a2-4192-9378-b5bc425693a2 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:04.021992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 981/1959] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:15:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:15:07.607665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056028f9-9687-4482-ab1f-acc27f8dca3a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:07.608246 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056028f9-9687-4482-ab1f-acc27f8dca3a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:07.608442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056028f9-9687-4482-ab1f-acc27f8dca3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:07.647152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-056028f9-9687-4482-ab1f-acc27f8dca3a None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:07.649842 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-056028f9-9687-4482-ab1f-acc27f8dca3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:07.661785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 979/1960] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:15:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:07.680412 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0b9c3-57a2-4192-9378-b5bc425693a2 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:07.695749 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0b9c3-57a2-4192-9378-b5bc425693a2 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:07.704027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6449373-d94b-4d97-a27f-b09c23afe823 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:07.705653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6449373-d94b-4d97-a27f-b09c23afe823 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:07.706467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6449373-d94b-4d97-a27f-b09c23afe823 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:07.706957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6449373-d94b-4d97-a27f-b09c23afe823 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:07.718166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6449373-d94b-4d97-a27f-b09c23afe823 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:07.721363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6449373-d94b-4d97-a27f-b09c23afe823 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:07.734314 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 982/1961] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:15:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:15:17.634620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=49 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:17.728318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=44 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:21.035219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695de17-ef67-4bc2-b1f6-8f4d8d82baa1 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:21.035792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695de17-ef67-4bc2-b1f6-8f4d8d82baa1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:21.035893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695de17-ef67-4bc2-b1f6-8f4d8d82baa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:21.054547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9695de17-ef67-4bc2-b1f6-8f4d8d82baa1 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:21.055769 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9695de17-ef67-4bc2-b1f6-8f4d8d82baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:21.063988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 980/1962] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 24 13:15:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:21.078601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6449373-d94b-4d97-a27f-b09c23afe823 service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:21.090598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6449373-d94b-4d97-a27f-b09c23afe823 service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:21.095763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40575a2-bae6-46ee-b525-4a58c8d32af8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:21.096374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40575a2-bae6-46ee-b525-4a58c8d32af8 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:21.096660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40575a2-bae6-46ee-b525-4a58c8d32af8 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:21.096940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40575a2-bae6-46ee-b525-4a58c8d32af8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:21.108607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40575a2-bae6-46ee-b525-4a58c8d32af8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:21.110285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40575a2-bae6-46ee-b525-4a58c8d32af8 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:21.118680 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 983/1963] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:15:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:15:23.150550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695de17-ef67-4bc2-b1f6-8f4d8d82baa1 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:23.153645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9695de17-ef67-4bc2-b1f6-8f4d8d82baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:23.169648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695de17-ef67-4bc2-b1f6-8f4d8d82baa1 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:23.174495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:23.175221 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.175546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.175941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] PATH_INFO: `/v3/users/1fd3d61c9fad4ac8ae26433ec77cd9c8` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.177113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd3d61c9fad4ac8ae26433ec77cd9c8)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:23.179174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:23.228405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd3d61c9fad4ac8ae26433ec77cd9c8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.241707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd3d61c9fad4ac8ae26433ec77cd9c8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.242415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd3d61c9fad4ac8ae26433ec77cd9c8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.251204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd3d61c9fad4ac8ae26433ec77cd9c8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.258586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d637a25c-f6d7-4d39-9ca4-f36e8e6ab9bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd3d61c9fad4ac8ae26433ec77cd9c8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.265987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 981/1964] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:15:23 2022] DELETE /identity/v3/users/1fd3d61c9fad4ac8ae26433ec77cd9c8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:15:23.439888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40575a2-bae6-46ee-b525-4a58c8d32af8 service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:23.452700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40575a2-bae6-46ee-b525-4a58c8d32af8 service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:23.511984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:23.512618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.512903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.513212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] PATH_INFO: `/v3/projects/5b028ba991e14292803f73d38f7003dc` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.514352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b028ba991e14292803f73d38f7003dc)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:23.516306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:23.562865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b028ba991e14292803f73d38f7003dc', 'request_id': 'req-7bfa6582-dafe-47b4-950c-007c903c778e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.568994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b028ba991e14292803f73d38f7003dc', 'request_id': 'req-7bfa6582-dafe-47b4-950c-007c903c778e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.576336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b028ba991e14292803f73d38f7003dc', 'request_id': 'req-7bfa6582-dafe-47b4-950c-007c903c778e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.586453 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] The token cache is being invalidate because project 5b028ba991e14292803f73d38f7003dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:15:23.586745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.587373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7bfa6582-dafe-47b4-950c-007c903c778e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:15:23.588906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 984/1965] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:15:23 2022] DELETE /identity/v3/projects/5b028ba991e14292803f73d38f7003dc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:15:23.594488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fc778-28bf-4bf8-ba09-4dfa053efd3b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.594783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fc778-28bf-4bf8-ba09-4dfa053efd3b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.595081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fc778-28bf-4bf8-ba09-4dfa053efd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.615810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-150fc778-28bf-4bf8-ba09-4dfa053efd3b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:23.617030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-150fc778-28bf-4bf8-ba09-4dfa053efd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:23.674988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 982/1966] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:23.684079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e181c3-1943-4ca1-8336-ea85a480a479 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.684370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e181c3-1943-4ca1-8336-ea85a480a479 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.684643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e181c3-1943-4ca1-8336-ea85a480a479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.704324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-83e181c3-1943-4ca1-8336-ea85a480a479 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:23.705506 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-83e181c3-1943-4ca1-8336-ea85a480a479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:23.714004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 985/1967] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:23.718312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fc778-28bf-4bf8-ba09-4dfa053efd3b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:23.727951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fc778-28bf-4bf8-ba09-4dfa053efd3b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:23.733313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:23.733917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.734201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.734477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.735152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:15:23.735458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:23.737052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:23.749083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 983/1968] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:15:23.753361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e181c3-1943-4ca1-8336-ea85a480a479 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:23.763090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e181c3-1943-4ca1-8336-ea85a480a479 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:23.767761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea926e-6100-4f7b-8d73-30a027d31f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:23.768356 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea926e-6100-4f7b-8d73-30a027d31f2b admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.768641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea926e-6100-4f7b-8d73-30a027d31f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.768918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea926e-6100-4f7b-8d73-30a027d31f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.769550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea926e-6100-4f7b-8d73-30a027d31f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:23.771500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea926e-6100-4f7b-8d73-30a027d31f2b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:23.798096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 986/1969] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:15:23 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:15:23.802176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:23.813094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a650e-f76c-4684-a5ce-7f90a1733725 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:23.862219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:23.875590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244840ee-1702-4b56-8680-111cd9f0bb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:23.876243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244840ee-1702-4b56-8680-111cd9f0bb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.876529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244840ee-1702-4b56-8680-111cd9f0bb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.876808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244840ee-1702-4b56-8680-111cd9f0bb87 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.877454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244840ee-1702-4b56-8680-111cd9f0bb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:23.879498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244840ee-1702-4b56-8680-111cd9f0bb87 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:23.905558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 984/1970] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:15:23 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:15:23.906829 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea926e-6100-4f7b-8d73-30a027d31f2b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:23.918682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea926e-6100-4f7b-8d73-30a027d31f2b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:23.923370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21760a0a-a87f-475a-9a40-e395cf7a8d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:23.924089 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21760a0a-a87f-475a-9a40-e395cf7a8d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.924433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21760a0a-a87f-475a-9a40-e395cf7a8d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.924735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21760a0a-a87f-475a-9a40-e395cf7a8d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.925380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21760a0a-a87f-475a-9a40-e395cf7a8d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:23.926995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21760a0a-a87f-475a-9a40-e395cf7a8d81 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:23.940624 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 987/1971] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:15:23.941724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244840ee-1702-4b56-8680-111cd9f0bb87 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:23.952333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244840ee-1702-4b56-8680-111cd9f0bb87 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:23.959029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f346d3c-4590-4636-a27a-b3dd24d4e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:23.959743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f346d3c-4590-4636-a27a-b3dd24d4e697 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.960142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f346d3c-4590-4636-a27a-b3dd24d4e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.960426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f346d3c-4590-4636-a27a-b3dd24d4e697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:23.961078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f346d3c-4590-4636-a27a-b3dd24d4e697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:23.962670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f346d3c-4590-4636-a27a-b3dd24d4e697 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:23.978656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 985/1972] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:15:23.980399 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21760a0a-a87f-475a-9a40-e395cf7a8d81 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:23.992420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21760a0a-a87f-475a-9a40-e395cf7a8d81 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:23.997212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:23.998008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:23.998345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:23.998625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] PATH_INFO: `/v3/projects/e6ab0fbf44394afda8dd8f49818ccc63/users/a769bcc0330a4171948a3cc930bd1ee2/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:24.014498 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ab0fbf44394afda8dd8f49818ccc63, user_id=a769bcc0330a4171948a3cc930bd1ee2, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:24.017352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:24.030979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 988/1973] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:15:23 2022] PUT /identity/v3/projects/e6ab0fbf44394afda8dd8f49818ccc63/users/a769bcc0330a4171948a3cc930bd1ee2/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:15:24.036036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6da0e9-8482-45e6-b551-55ea70a45c13 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:24.036329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6da0e9-8482-45e6-b551-55ea70a45c13 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:24.036609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6da0e9-8482-45e6-b551-55ea70a45c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:24.056808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-3a6da0e9-8482-45e6-b551-55ea70a45c13 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:24.058293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3a6da0e9-8482-45e6-b551-55ea70a45c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:24.120969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 986/1974] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:24.130553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:24.132272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:24.146893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:24.191197 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1876c8-038c-4481-a741-d1ce13e17993 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:24.194863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10fe75-8f0b-47d4-9d3a-74cfb03edd47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:24.195507 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fe75-8f0b-47d4-9d3a-74cfb03edd47 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:24.195788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fe75-8f0b-47d4-9d3a-74cfb03edd47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:24.196061 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10fe75-8f0b-47d4-9d3a-74cfb03edd47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:24.204973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fe75-8f0b-47d4-9d3a-74cfb03edd47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:24.206385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10fe75-8f0b-47d4-9d3a-74cfb03edd47 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:24.216255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 989/1975] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:15:28.295520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e419-6dca-4885-bb98-4beed9d31c31 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:28.295965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e419-6dca-4885-bb98-4beed9d31c31 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:28.296225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5e419-6dca-4885-bb98-4beed9d31c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:28.336500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-73d5e419-6dca-4885-bb98-4beed9d31c31 None None] MFA Rules not processed for user `a769bcc0330a4171948a3cc930bd1ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:28.339209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-73d5e419-6dca-4885-bb98-4beed9d31c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:28.440043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 987/1976] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:15:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:28.444549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10fe75-8f0b-47d4-9d3a-74cfb03edd47 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:28.456001 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10fe75-8f0b-47d4-9d3a-74cfb03edd47 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:28.461070 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952982b-1509-4d7a-b46e-c21aff2f577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:28.461682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952982b-1509-4d7a-b46e-c21aff2f577a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:28.461966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952982b-1509-4d7a-b46e-c21aff2f577a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:28.462243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952982b-1509-4d7a-b46e-c21aff2f577a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:28.462893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952982b-1509-4d7a-b46e-c21aff2f577a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:28.465608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952982b-1509-4d7a-b46e-c21aff2f577a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:28.497172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 990/1977] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:15:28 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:15:28.500910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e419-6dca-4885-bb98-4beed9d31c31 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:28.513354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5e419-6dca-4885-bb98-4beed9d31c31 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:28.580235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853683e-ad17-4968-99c0-884e6de0fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:28.580445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853683e-ad17-4968-99c0-884e6de0fef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:28.580619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853683e-ad17-4968-99c0-884e6de0fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:28.580902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853683e-ad17-4968-99c0-884e6de0fef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:28.581560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853683e-ad17-4968-99c0-884e6de0fef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:28.583537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853683e-ad17-4968-99c0-884e6de0fef0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:28.595213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:28.614507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 988/1978] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:15:28 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:15:28.618541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952982b-1509-4d7a-b46e-c21aff2f577a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:28.630532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952982b-1509-4d7a-b46e-c21aff2f577a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:28.635907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb448c-9618-4f57-884c-4c8177a48dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:28.636527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb448c-9618-4f57-884c-4c8177a48dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:28.637422 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb448c-9618-4f57-884c-4c8177a48dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:28.637422 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb448c-9618-4f57-884c-4c8177a48dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:28.637727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb448c-9618-4f57-884c-4c8177a48dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:28.639716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bb448c-9618-4f57-884c-4c8177a48dd7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:28.651285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 991/1979] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:15:28.655650 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853683e-ad17-4968-99c0-884e6de0fef0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:28.666114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853683e-ad17-4968-99c0-884e6de0fef0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:28.670870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c061d29-1e6a-4df3-9b7a-c9ab424bfdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:28.671699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c061d29-1e6a-4df3-9b7a-c9ab424bfdea admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:28.671982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c061d29-1e6a-4df3-9b7a-c9ab424bfdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:28.672257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c061d29-1e6a-4df3-9b7a-c9ab424bfdea admin admin] PATH_INFO: `/v3/projects/aa714009ed494760aba72453db740147/users/8e6556d4eaa44d66a52fe63a521523d4/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:28.690040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c061d29-1e6a-4df3-9b7a-c9ab424bfdea admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa714009ed494760aba72453db740147, user_id=8e6556d4eaa44d66a52fe63a521523d4, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:28.692815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c061d29-1e6a-4df3-9b7a-c9ab424bfdea admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:28.707447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 989/1980] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:15:28 2022] PUT /identity/v3/projects/aa714009ed494760aba72453db740147/users/8e6556d4eaa44d66a52fe63a521523d4/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:15:28.711617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb448c-9618-4f57-884c-4c8177a48dd7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:28.724715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb448c-9618-4f57-884c-4c8177a48dd7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:28.796336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:28.796958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:28.797268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:28.797532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:28.798186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:28.799835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:28.820586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 992/1981] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:15:28.821780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c061d29-1e6a-4df3-9b7a-c9ab424bfdea admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:28.862199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c061d29-1e6a-4df3-9b7a-c9ab424bfdea admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:28.867021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf7ef-1a32-4a3e-8878-2e336fe2bb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:28.867723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf7ef-1a32-4a3e-8878-2e336fe2bb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:28.868007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf7ef-1a32-4a3e-8878-2e336fe2bb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:28.868280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf7ef-1a32-4a3e-8878-2e336fe2bb28 admin admin] PATH_INFO: `/v3/projects/aa714009ed494760aba72453db740147/users/8e6556d4eaa44d66a52fe63a521523d4/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:28.871319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf7ef-1a32-4a3e-8878-2e336fe2bb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa714009ed494760aba72453db740147, user_id=8e6556d4eaa44d66a52fe63a521523d4, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:28.873955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf7ef-1a32-4a3e-8878-2e336fe2bb28 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:28.892391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 990/1982] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:15:28 2022] PUT /identity/v3/projects/aa714009ed494760aba72453db740147/users/8e6556d4eaa44d66a52fe63a521523d4/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:15:29.000194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:29.002022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:29.123122 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5159598-6313-441f-a748-6775776d4970 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:29.147452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:29.242100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:29.242740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:29.243323 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:29.244888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:29.246899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:29.260932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:29.262451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:29.274890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 993/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:15:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:15:34.602159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875416e-0518-428c-9d65-e002b8c5da17 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:34.602631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875416e-0518-428c-9d65-e002b8c5da17 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:34.602712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875416e-0518-428c-9d65-e002b8c5da17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:34.639276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-2875416e-0518-428c-9d65-e002b8c5da17 None None] MFA Rules not processed for user `8e6556d4eaa44d66a52fe63a521523d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:34.640665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2875416e-0518-428c-9d65-e002b8c5da17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:34.731226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 991/1984] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:15:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:34.743656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:34.745902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:34.831566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469ec42-d322-4ddc-bfe2-54a178ffe277 service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:34.835834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d0fc260e254d2d80c6927c52cd8a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:34.836515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:34.836801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:34.837079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:34.955150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:34.956603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:34.983151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 994/1985] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:15:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:15:35.128154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8eb91-e06d-46fe-bee5-3796d3c62508 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:35.128154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8eb91-e06d-46fe-bee5-3796d3c62508 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:35.129523 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8eb91-e06d-46fe-bee5-3796d3c62508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:35.154884 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-58f8eb91-e06d-46fe-bee5-3796d3c62508 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:35.156157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-58f8eb91-e06d-46fe-bee5-3796d3c62508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:35.220432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 992/1986] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:15:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:35.248947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:35.251199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:35.343103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eca3f1-2e2f-444b-8b2d-9a7e591ed10d service glance] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:35.348452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9ef8d-a218-4c0c-a509-224be1538c5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:35.349068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9ef8d-a218-4c0c-a509-224be1538c5a service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:35.349351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9ef8d-a218-4c0c-a509-224be1538c5a service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:35.349625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9ef8d-a218-4c0c-a509-224be1538c5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:35.356982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:35.359903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9ef8d-a218-4c0c-a509-224be1538c5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:35.362130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9ef8d-a218-4c0c-a509-224be1538c5a service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:35.371093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 995/1987] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:15:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:15:42.533352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b171-fe93-4fd1-859f-f281a452d300 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:42.533864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b171-fe93-4fd1-859f-f281a452d300 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:42.533969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d8b171-fe93-4fd1-859f-f281a452d300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:42.558212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a6d8b171-fe93-4fd1-859f-f281a452d300 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:42.559960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a6d8b171-fe93-4fd1-859f-f281a452d300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:42.568598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 993/1988] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 24 13:15:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:42.581922 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9ef8d-a218-4c0c-a509-224be1538c5a service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:42.593338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9ef8d-a218-4c0c-a509-224be1538c5a service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:42.600731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074072d2-44ae-4cf4-a4aa-615817a5c812 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:42.601366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074072d2-44ae-4cf4-a4aa-615817a5c812 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:42.601725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074072d2-44ae-4cf4-a4aa-615817a5c812 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:42.602079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074072d2-44ae-4cf4-a4aa-615817a5c812 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:42.612174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074072d2-44ae-4cf4-a4aa-615817a5c812 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:42.614384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074072d2-44ae-4cf4-a4aa-615817a5c812 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:42.624833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 996/1989] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:15:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:15:45.568353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c88395-05f3-4b32-9be6-ddc2e8623ece None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:45.568846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c88395-05f3-4b32-9be6-ddc2e8623ece None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:45.568953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c88395-05f3-4b32-9be6-ddc2e8623ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:45.589143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-66c88395-05f3-4b32-9be6-ddc2e8623ece None None] MFA Rules not processed for user `a769bcc0330a4171948a3cc930bd1ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:45.590322 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-66c88395-05f3-4b32-9be6-ddc2e8623ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:45.597088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 994/1990] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:15:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:15:48.180131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35679-a160-43e1-8595-574983b20f15 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:48.180597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35679-a160-43e1-8595-574983b20f15 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:48.180733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a35679-a160-43e1-8595-574983b20f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:48.199953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-05a35679-a160-43e1-8595-574983b20f15 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:15:48.201163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-05a35679-a160-43e1-8595-574983b20f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:15:48.208720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 997/1991] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:15:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:15:48.222009 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c88395-05f3-4b32-9be6-ddc2e8623ece None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:15:48.234454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c88395-05f3-4b32-9be6-ddc2e8623ece None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:15:48.239595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f909fd5-a358-4ce5-93cc-29f591a1016d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:15:48.240237 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f909fd5-a358-4ce5-93cc-29f591a1016d service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:15:48.240520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f909fd5-a358-4ce5-93cc-29f591a1016d service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:15:48.240797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f909fd5-a358-4ce5-93cc-29f591a1016d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:15:48.249361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f909fd5-a358-4ce5-93cc-29f591a1016d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:15:48.250956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f909fd5-a358-4ce5-93cc-29f591a1016d service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:15:48.259452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 995/1992] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:15:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:15:58.194861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=11 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:15:58.256016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:INSERT=4,keystone:SELECT=88 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:16:15.865646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727c5a5-81cc-4567-baea-ab687b93a39f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:16:15.866216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727c5a5-81cc-4567-baea-ab687b93a39f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:16:15.866316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727c5a5-81cc-4567-baea-ab687b93a39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:16:15.886598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b727c5a5-81cc-4567-baea-ab687b93a39f None None] MFA Rules not processed for user `abd7bde497c14d25a06c276609aa143f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:16:15.887788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b727c5a5-81cc-4567-baea-ab687b93a39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:16:15.944112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 998/1993] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:16:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:16:25.621674 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457db47-e420-4fb9-bf0c-cb3d4f97dfea None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:16:25.622253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457db47-e420-4fb9-bf0c-cb3d4f97dfea None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:16:25.622253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457db47-e420-4fb9-bf0c-cb3d4f97dfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:16:25.643424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c457db47-e420-4fb9-bf0c-cb3d4f97dfea None None] MFA Rules not processed for user `c5619c8183e04e1d9073f1b40e057967`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:16:25.644623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c457db47-e420-4fb9-bf0c-cb3d4f97dfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:16:25.700826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 996/1994] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:16:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:16:25.936293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=25 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:16:27.599715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727c5a5-81cc-4567-baea-ab687b93a39f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:16:27.601620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b727c5a5-81cc-4567-baea-ab687b93a39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:16:27.668288 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727c5a5-81cc-4567-baea-ab687b93a39f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:16:27.671893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9906173-ab7d-4270-852e-b270fdb15039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:16:27.672504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9906173-ab7d-4270-852e-b270fdb15039 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:16:27.672790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9906173-ab7d-4270-852e-b270fdb15039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:16:27.673076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9906173-ab7d-4270-852e-b270fdb15039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:16:27.683345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9906173-ab7d-4270-852e-b270fdb15039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:16:27.685053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9906173-ab7d-4270-852e-b270fdb15039 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:16:27.697018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 999/1995] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:16:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:16:35.693673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=25 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:16:36.786212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efcbd8-6057-4632-842a-ee395835f10b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:16:36.786800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efcbd8-6057-4632-842a-ee395835f10b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:16:36.786800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efcbd8-6057-4632-842a-ee395835f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:16:36.807304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f2efcbd8-6057-4632-842a-ee395835f10b None None] MFA Rules not processed for user `d29ff25b062346f6bcf9910b1fa8db5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:16:36.808521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f2efcbd8-6057-4632-842a-ee395835f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:16:36.862574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 997/1996] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:16:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:16:37.691986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=26 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:16:46.856021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=25 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:16:56.772124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8954-b1a8-4a8c-84bb-7c957e28eb39 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:16:56.772628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8954-b1a8-4a8c-84bb-7c957e28eb39 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:16:56.772741 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8954-b1a8-4a8c-84bb-7c957e28eb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:16:56.792103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e2bd8954-b1a8-4a8c-84bb-7c957e28eb39 None None] MFA Rules not processed for user `3e02f6f701324af6b53762569f1a9d21`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:16:56.793323 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e2bd8954-b1a8-4a8c-84bb-7c957e28eb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:16:56.849210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1000/1997] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:16:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:06.842285 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=25 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:20.572472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efcbd8-6057-4632-842a-ee395835f10b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:20.582124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efcbd8-6057-4632-842a-ee395835f10b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:20.585936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:20.586673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:20.586960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:20.587259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:20.589159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:20.668779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:20.670278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:20.682358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 998/1998] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:21.879229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8954-b1a8-4a8c-84bb-7c957e28eb39 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:21.890238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd8954-b1a8-4a8c-84bb-7c957e28eb39 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:21.894411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:21.895046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:21.895347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:21.895620 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:21.897363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:21.913079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:21.914517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:21.926140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1001/1999] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:24.640866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:24.642479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:24.712982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecd77d-6c4b-4249-867a-17e01f2443d0 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:24.717772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:24.718412 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:24.718697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:24.718979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] PATH_INFO: `/v3/users/abd7bde497c14d25a06c276609aa143f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:24.720138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd7bde497c14d25a06c276609aa143f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:24.722099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:24.768806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd7bde497c14d25a06c276609aa143f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:24.780482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd7bde497c14d25a06c276609aa143f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:24.781148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd7bde497c14d25a06c276609aa143f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:24.796144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd7bde497c14d25a06c276609aa143f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:24.805063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd7bde497c14d25a06c276609aa143f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:24.810903 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 999/2000] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:17:24 2022] DELETE /identity/v3/users/abd7bde497c14d25a06c276609aa143f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:17:24.988826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:25.000200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26cafe-afeb-45e6-8f50-ce326183d8a0 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:25.059293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:25.059856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:25.060143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:25.060424 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] PATH_INFO: `/v3/projects/f027aba5a38b422fbc30d64b9ee2ef79` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:25.061546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f027aba5a38b422fbc30d64b9ee2ef79)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:25.063501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:25.111554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f027aba5a38b422fbc30d64b9ee2ef79', 'request_id': 'req-86bca61a-36fc-475b-9850-50a41eb0741a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:25.117909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f027aba5a38b422fbc30d64b9ee2ef79', 'request_id': 'req-86bca61a-36fc-475b-9850-50a41eb0741a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:25.125315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f027aba5a38b422fbc30d64b9ee2ef79', 'request_id': 'req-86bca61a-36fc-475b-9850-50a41eb0741a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:25.137597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] The token cache is being invalidate because project f027aba5a38b422fbc30d64b9ee2ef79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:17:25.137911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:25.138528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:17:25.140024 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1002/2001] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:17:24 2022] DELETE /identity/v3/projects/f027aba5a38b422fbc30d64b9ee2ef79 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:17:32.479645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:32.481270 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:32.487826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:32.560248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526489cc-a8e5-4d54-9de3-93928f30f549 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:32.623867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f95b41-e8e8-4fe1-ae0c-9404000deef9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:32.624481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95b41-e8e8-4fe1-ae0c-9404000deef9 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:32.624766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95b41-e8e8-4fe1-ae0c-9404000deef9 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:32.625041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f95b41-e8e8-4fe1-ae0c-9404000deef9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:32.626895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-32f95b41-e8e8-4fe1-ae0c-9404000deef9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:32.684470 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f95b41-e8e8-4fe1-ae0c-9404000deef9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:32.686055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f95b41-e8e8-4fe1-ae0c-9404000deef9 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:32.695426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1000/2002] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:35.134363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:36.395169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:36.397347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:36.454601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bca61a-36fc-475b-9850-50a41eb0741a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:36.458232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:36.458838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:36.459147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:36.460365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:36.461896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:36.519160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:36.520618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:36.531714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1003/2003] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:42.690778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=82 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:46.527208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=48 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:17:49.093543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8fbf4-2263-40d7-af02-072e8ae726f5 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:49.094046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8fbf4-2263-40d7-af02-072e8ae726f5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:49.094135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8fbf4-2263-40d7-af02-072e8ae726f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:49.113581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-c9c8fbf4-2263-40d7-af02-072e8ae726f5 None None] MFA Rules not processed for user `c5619c8183e04e1d9073f1b40e057967`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:17:49.114781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c9c8fbf4-2263-40d7-af02-072e8ae726f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:49.122200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1001/2004] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:17:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:17:49.440897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:49.442366 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:49.505052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6497c83-b870-4643-a5fd-92685f94408f service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:49.510243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:49.511137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:49.511442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:49.511717 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:49.513543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:49.523956 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:49.525392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:49.539462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1004/2005] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:49.857668 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8fbf4-2263-40d7-af02-072e8ae726f5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:49.859143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c9c8fbf4-2263-40d7-af02-072e8ae726f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:49.940181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8fbf4-2263-40d7-af02-072e8ae726f5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:49.945242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:49.945853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:49.946140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:49.946416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:49.948160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:49.959746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:49.961367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:49.971372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1002/2006] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:53.259198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:53.261212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:53.278974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cbfd1-ff30-4430-b3aa-3f83c291dbdd service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:53.286480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:53.288137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:53.288604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:53.289677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:53.292460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:53.310778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:53.312593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:53.322179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1005/2007] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:53.968524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:53.970236 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:53.990119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0681d202-dda0-4d7d-b41d-40b42514bbd6 service placement] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:53.996349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:53.996973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:53.997255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:53.997532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:53.999300 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:54.012396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:54.014023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:54.025690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1003/2008] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:17:55.529154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:17:55.542778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39b7ef-d3e7-46bf-a6cf-41c66a43c676 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:17:55.548181 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:17:55.548782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:17:55.549052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:17:55.549326 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:17:55.551016 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:17:55.612571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:17:55.614175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:17:55.624129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1006/2009] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:17:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:00.315574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:00.328510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ec461-7f97-40ca-9510-95b50e01578c service nova] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:00.334913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:00.335643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:00.335999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:00.336362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:00.338319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:00.351657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:00.353363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:00.366385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1004/2010] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:01.503441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:01.513310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0b90f-d28a-464b-bed8-90b938c2e27b service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:01.516875 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:01.517470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:01.517988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:01.518264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:01.520057 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:01.588062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:01.589504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:01.599345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1007/2011] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:04.033131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:04.065501 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cd25-81a6-4449-a870-174b1cf8bf44 service nova] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:04.073003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:04.073901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:04.074594 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:04.075039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:04.077112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:04.094261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:04.096491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:04.119017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1005/2012] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:05.988390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:05.990037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:06.015696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5abd2-6412-4c4c-b593-8f403c459dca service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:06.020736 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:06.022455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:06.022950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:06.023461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] PATH_INFO: `/v3/users/d29ff25b062346f6bcf9910b1fa8db5c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:06.024927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d29ff25b062346f6bcf9910b1fa8db5c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:06.027003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:06.050460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:06.107233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd29ff25b062346f6bcf9910b1fa8db5c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:06.122864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd29ff25b062346f6bcf9910b1fa8db5c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:06.124009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd29ff25b062346f6bcf9910b1fa8db5c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:06.145852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd29ff25b062346f6bcf9910b1fa8db5c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:06.148755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-bfc70a09-232e-4408-9c76-8b401beb0a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd29ff25b062346f6bcf9910b1fa8db5c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:06.168297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1008/2013] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:05 2022] DELETE /identity/v3/users/d29ff25b062346f6bcf9910b1fa8db5c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:06.921739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:06.939239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf6dfa-962d-4728-ad7a-120debf6b3d6 service nova] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:07.030178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:07.031239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.031581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.032064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] PATH_INFO: `/v3/projects/85d62dec32104e7b8c595953a479026b` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.033273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d62dec32104e7b8c595953a479026b)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:07.035295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:07.096543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:07.100610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d62dec32104e7b8c595953a479026b', 'request_id': 'req-50de78a7-6218-414b-8efb-2bb8b3f08564'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:07.109939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d62dec32104e7b8c595953a479026b', 'request_id': 'req-50de78a7-6218-414b-8efb-2bb8b3f08564'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:07.120216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d62dec32104e7b8c595953a479026b', 'request_id': 'req-50de78a7-6218-414b-8efb-2bb8b3f08564'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:07.136820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] The token cache is being invalidate because project 85d62dec32104e7b8c595953a479026b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:07.137112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:07.137993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-50de78a7-6218-414b-8efb-2bb8b3f08564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:07.140167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1006/2014] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:06 2022] DELETE /identity/v3/projects/85d62dec32104e7b8c595953a479026b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:07.144883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fedeb-8315-4407-8232-c551a12b1d07 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.146145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fedeb-8315-4407-8232-c551a12b1d07 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.146833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2fedeb-8315-4407-8232-c551a12b1d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.172176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db2fedeb-8315-4407-8232-c551a12b1d07 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:07.173719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db2fedeb-8315-4407-8232-c551a12b1d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:07.266171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1009/2015] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:07.274765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9548b2-fb8c-4ce7-96ca-61c6f1f48f13 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.275144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9548b2-fb8c-4ce7-96ca-61c6f1f48f13 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.275355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9548b2-fb8c-4ce7-96ca-61c6f1f48f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.297452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9b9548b2-fb8c-4ce7-96ca-61c6f1f48f13 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:07.298697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9b9548b2-fb8c-4ce7-96ca-61c6f1f48f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:07.308970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1007/2016] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:07.313222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fedeb-8315-4407-8232-c551a12b1d07 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:07.324598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2fedeb-8315-4407-8232-c551a12b1d07 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:07.329246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:07.329879 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.330161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.330434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.331132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:18:07.331441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:07.333091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:07.346766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1010/2017] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:18:07.352578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9548b2-fb8c-4ce7-96ca-61c6f1f48f13 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:07.362446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9548b2-fb8c-4ce7-96ca-61c6f1f48f13 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:07.367519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b2dff-4fa3-4bc9-abd2-ff5f3f99060a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:07.368148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2dff-4fa3-4bc9-abd2-ff5f3f99060a admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.368429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2dff-4fa3-4bc9-abd2-ff5f3f99060a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.368710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b2dff-4fa3-4bc9-abd2-ff5f3f99060a admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.371093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b2dff-4fa3-4bc9-abd2-ff5f3f99060a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:07.372008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b2dff-4fa3-4bc9-abd2-ff5f3f99060a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:07.406557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1008/2018] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:18:07 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:18:07.408393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:07.420644 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d49cd0-8884-471a-bf9c-742a04e04783 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:07.486781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aadaa2-ed15-407d-9b36-78acbcc63d65 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.487078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aadaa2-ed15-407d-9b36-78acbcc63d65 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.487379 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aadaa2-ed15-407d-9b36-78acbcc63d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.507990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e16c-89ae-4869-bfb0-2faa9a7b7f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:07.508621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e16c-89ae-4869-bfb0-2faa9a7b7f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.508905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e16c-89ae-4869-bfb0-2faa9a7b7f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.509182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e16c-89ae-4869-bfb0-2faa9a7b7f20 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.509813 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e16c-89ae-4869-bfb0-2faa9a7b7f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:07.513017 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e16c-89ae-4869-bfb0-2faa9a7b7f20 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:07.513346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a2aadaa2-ed15-407d-9b36-78acbcc63d65 None None] MFA Rules not processed for user `3f69df43e6ca4663a9ef819c0331c13d`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:07.516041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a2aadaa2-ed15-407d-9b36-78acbcc63d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:07.544542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1011/2019] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:18:07 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:07.548691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e16c-89ae-4869-bfb0-2faa9a7b7f20 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:07.562494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e16c-89ae-4869-bfb0-2faa9a7b7f20 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:07.569422 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b2932-9829-4d26-a999-68f1a646ad67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:07.570052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b2932-9829-4d26-a999-68f1a646ad67 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.570334 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b2932-9829-4d26-a999-68f1a646ad67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.570611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b2932-9829-4d26-a999-68f1a646ad67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.571293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b2932-9829-4d26-a999-68f1a646ad67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:07.573080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b2932-9829-4d26-a999-68f1a646ad67 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:07.578971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:07.590745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1012/2020] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:07.591981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b2932-9829-4d26-a999-68f1a646ad67 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:07.604780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b2932-9829-4d26-a999-68f1a646ad67 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:07.607253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1009/2021] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:07.614485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd749da-eac2-42ee-a989-f03be399d63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:07.615925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd749da-eac2-42ee-a989-f03be399d63c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.616249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd749da-eac2-42ee-a989-f03be399d63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.616531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd749da-eac2-42ee-a989-f03be399d63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.617351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd749da-eac2-42ee-a989-f03be399d63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:07.618936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd749da-eac2-42ee-a989-f03be399d63c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:07.634808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1013/2022] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:07.635485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aadaa2-ed15-407d-9b36-78acbcc63d65 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:07.665531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aadaa2-ed15-407d-9b36-78acbcc63d65 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:07.680956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:07.681443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.682350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.682531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] PATH_INFO: `/v3/projects/c4a65ae892634c7b99d402d482d6778a/users/ea9e1b02082340ad9eedd2582bba9a3e/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.702989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4a65ae892634c7b99d402d482d6778a, user_id=ea9e1b02082340ad9eedd2582bba9a3e, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:07.705782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:07.733504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1010/2023] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:07 2022] PUT /identity/v3/projects/c4a65ae892634c7b99d402d482d6778a/users/ea9e1b02082340ad9eedd2582bba9a3e/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:18:07.736005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c024048-a144-4bf3-88d2-d8b2ba54abd5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.736530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c024048-a144-4bf3-88d2-d8b2ba54abd5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.737069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c024048-a144-4bf3-88d2-d8b2ba54abd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.776714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8c024048-a144-4bf3-88d2-d8b2ba54abd5 None None] MFA Rules not processed for user `ea9e1b02082340ad9eedd2582bba9a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:07.778239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8c024048-a144-4bf3-88d2-d8b2ba54abd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:07.873363 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1014/2024] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:07.885896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:07.888276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:07.968182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ef42d-7e52-4ba0-9ffe-4ac95f729f88 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:07.976276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ea67e-9b8a-45e0-8acf-5529b7ac10f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:07.976992 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ea67e-9b8a-45e0-8acf-5529b7ac10f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:07.977350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ea67e-9b8a-45e0-8acf-5529b7ac10f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:07.978140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ea67e-9b8a-45e0-8acf-5529b7ac10f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:07.988173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ea67e-9b8a-45e0-8acf-5529b7ac10f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:07.990339 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ea67e-9b8a-45e0-8acf-5529b7ac10f0 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:08.001535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1011/2025] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:18:16.980735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c024048-a144-4bf3-88d2-d8b2ba54abd5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:17.067224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c024048-a144-4bf3-88d2-d8b2ba54abd5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:17.072747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:17.073761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:17.074220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:17.074681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] PATH_INFO: `/v3/users/ea9e1b02082340ad9eedd2582bba9a3e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:17.076487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9e1b02082340ad9eedd2582bba9a3e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:17.079540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:17.140882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9e1b02082340ad9eedd2582bba9a3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.154313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9e1b02082340ad9eedd2582bba9a3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.155044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9e1b02082340ad9eedd2582bba9a3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.162910 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9e1b02082340ad9eedd2582bba9a3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.163712 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9e1b02082340ad9eedd2582bba9a3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.181838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:17.188786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1015/2026] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:16 2022] DELETE /identity/v3/users/ea9e1b02082340ad9eedd2582bba9a3e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:17.193763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f58dc-78ce-4722-81b0-ba6e0f998f80 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:17.194312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f58dc-78ce-4722-81b0-ba6e0f998f80 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:17.196273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f58dc-78ce-4722-81b0-ba6e0f998f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:17.225730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1f5f58dc-78ce-4722-81b0-ba6e0f998f80 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:17.227195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1f5f58dc-78ce-4722-81b0-ba6e0f998f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:17.258298 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:17.323634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1012/2027] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:17.333858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:17.335349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:17.397869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35217bce-1beb-492a-8cca-13007988433d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:17.401562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ee3c9-8d4f-4e5b-a846-4cfe770a08bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:17.402184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ee3c9-8d4f-4e5b-a846-4cfe770a08bf service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:17.402467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ee3c9-8d4f-4e5b-a846-4cfe770a08bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:17.402752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ee3c9-8d4f-4e5b-a846-4cfe770a08bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:17.412818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ee3c9-8d4f-4e5b-a846-4cfe770a08bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:17.414265 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ee3c9-8d4f-4e5b-a846-4cfe770a08bf service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:17.424255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1016/2028] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:18:17.717583 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f58dc-78ce-4722-81b0-ba6e0f998f80 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:17.719031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1f5f58dc-78ce-4722-81b0-ba6e0f998f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:17.738855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f58dc-78ce-4722-81b0-ba6e0f998f80 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:17.744570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:17.745281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:17.745626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:17.745970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] PATH_INFO: `/v3/projects/c4a65ae892634c7b99d402d482d6778a` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:17.748401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4a65ae892634c7b99d402d482d6778a)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:17.751363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:17.808183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4a65ae892634c7b99d402d482d6778a', 'request_id': 'req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.820679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4a65ae892634c7b99d402d482d6778a', 'request_id': 'req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.830167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4a65ae892634c7b99d402d482d6778a', 'request_id': 'req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.842436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] The token cache is being invalidate because project c4a65ae892634c7b99d402d482d6778a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:17.842733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.843483 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6c5e7b3c-65b1-4f0c-8430-bfcd0f601fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:17.845371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1013/2029] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:17 2022] DELETE /identity/v3/projects/c4a65ae892634c7b99d402d482d6778a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:17.851330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce9d0f-a04a-4649-97e5-c04c174d5ce7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:17.851723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce9d0f-a04a-4649-97e5-c04c174d5ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:17.852079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce9d0f-a04a-4649-97e5-c04c174d5ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:17.877547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c2ce9d0f-a04a-4649-97e5-c04c174d5ce7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:17.878950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c2ce9d0f-a04a-4649-97e5-c04c174d5ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:17.894091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675a392-b444-446a-9586-787945da0eed None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:17.894466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675a392-b444-446a-9586-787945da0eed None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:17.894831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675a392-b444-446a-9586-787945da0eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:17.919939 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6675a392-b444-446a-9586-787945da0eed None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:17.921801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6675a392-b444-446a-9586-787945da0eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:17.972388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1017/2030] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:17.973876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7453f0f-2889-4025-bd07-666acfa34fef None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:17.974205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7453f0f-2889-4025-bd07-666acfa34fef None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:17.974486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7453f0f-2889-4025-bd07-666acfa34fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:17.995138 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a7453f0f-2889-4025-bd07-666acfa34fef None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:17.996705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a7453f0f-2889-4025-bd07-666acfa34fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:18.003963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1014/2031] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:18:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:18.005885 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f5dab-1135-4660-ad33-24929a23ad2a None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.006125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f5dab-1135-4660-ad33-24929a23ad2a None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.006481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f5dab-1135-4660-ad33-24929a23ad2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.008734 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1018/2032] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 24 13:18:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:18.037327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-690f5dab-1135-4660-ad33-24929a23ad2a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:18.038399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-690f5dab-1135-4660-ad33-24929a23ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:18.047855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1015/2033] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:18.052827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7453f0f-2889-4025-bd07-666acfa34fef None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:18.083923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7453f0f-2889-4025-bd07-666acfa34fef None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:18.088764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f5dab-1135-4660-ad33-24929a23ad2a None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:18.090192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-690f5dab-1135-4660-ad33-24929a23ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:18.090839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:18.091673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.092407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.092827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.093701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:18:18.094157 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:18.096297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:18.109781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1019/2034] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:18:18.111239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:18.113212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:18.167341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:18.195003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f5dab-1135-4660-ad33-24929a23ad2a None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:18.201319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed43ec3-2c6e-4296-ac8a-a01abe625095 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:18.201947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43ec3-2c6e-4296-ac8a-a01abe625095 service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.202229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43ec3-2c6e-4296-ac8a-a01abe625095 service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.202503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed43ec3-2c6e-4296-ac8a-a01abe625095 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.215207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bbbf8-b1d5-43f5-9fea-a18d6b843d3d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:18.215649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed43ec3-2c6e-4296-ac8a-a01abe625095 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:18.218060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed43ec3-2c6e-4296-ac8a-a01abe625095 service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:18.225930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cc7b1-bea1-422b-87d7-07fa6e943b33 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:18.226622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cc7b1-bea1-422b-87d7-07fa6e943b33 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.226912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cc7b1-bea1-422b-87d7-07fa6e943b33 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.227230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cc7b1-bea1-422b-87d7-07fa6e943b33 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.229760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1016/2035] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:18:18.230706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed43ec3-2c6e-4296-ac8a-a01abe625095 service swift] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:18.240938 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cc7b1-bea1-422b-87d7-07fa6e943b33 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:18.242966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cc7b1-bea1-422b-87d7-07fa6e943b33 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:18.244712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed43ec3-2c6e-4296-ac8a-a01abe625095 service swift] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:18.254729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51465057-aea5-42e1-a48a-ce0409500f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:18.257463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51465057-aea5-42e1-a48a-ce0409500f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.259495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51465057-aea5-42e1-a48a-ce0409500f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.259495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51465057-aea5-42e1-a48a-ce0409500f15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.260524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51465057-aea5-42e1-a48a-ce0409500f15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:18.261959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1020/2036] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:18:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:18:18.264085 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51465057-aea5-42e1-a48a-ce0409500f15 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:18.339892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1017/2037] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:18:18 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:18.343959 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cc7b1-bea1-422b-87d7-07fa6e943b33 service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:18.357904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cc7b1-bea1-422b-87d7-07fa6e943b33 service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:18.369676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:18.524639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659551c8-0933-44a4-8678-5a599b0090e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:18.525292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659551c8-0933-44a4-8678-5a599b0090e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.525581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659551c8-0933-44a4-8678-5a599b0090e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.526689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659551c8-0933-44a4-8678-5a599b0090e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.529192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659551c8-0933-44a4-8678-5a599b0090e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:18.531391 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659551c8-0933-44a4-8678-5a599b0090e3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:18.588759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1021/2038] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:18:18 2022] POST /identity/v3/users => generated 285 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:18:18.592946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51465057-aea5-42e1-a48a-ce0409500f15 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:18.611503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51465057-aea5-42e1-a48a-ce0409500f15 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:18.618361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681eb2ed-e6fb-4301-8862-8f2aa153e3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:18.618996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681eb2ed-e6fb-4301-8862-8f2aa153e3b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.619333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681eb2ed-e6fb-4301-8862-8f2aa153e3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.619613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681eb2ed-e6fb-4301-8862-8f2aa153e3b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.620343 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681eb2ed-e6fb-4301-8862-8f2aa153e3b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:18.623073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681eb2ed-e6fb-4301-8862-8f2aa153e3b2 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:18.639486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1018/2039] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:18.643183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659551c8-0933-44a4-8678-5a599b0090e3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:18.657210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659551c8-0933-44a4-8678-5a599b0090e3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:18.671023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f1a56-e1c8-4dbd-88d9-226db785b889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:18.672063 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1a56-e1c8-4dbd-88d9-226db785b889 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.672658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1a56-e1c8-4dbd-88d9-226db785b889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.673090 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f1a56-e1c8-4dbd-88d9-226db785b889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.673981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f1a56-e1c8-4dbd-88d9-226db785b889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:18.676543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f1a56-e1c8-4dbd-88d9-226db785b889 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:18.690958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1022/2040] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:18.696300 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681eb2ed-e6fb-4301-8862-8f2aa153e3b2 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:18.720098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681eb2ed-e6fb-4301-8862-8f2aa153e3b2 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:18.727415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cd3f9-bf2b-41e1-9cdc-76afdee2e3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:18.728375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cd3f9-bf2b-41e1-9cdc-76afdee2e3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.728767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cd3f9-bf2b-41e1-9cdc-76afdee2e3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.729100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cd3f9-bf2b-41e1-9cdc-76afdee2e3f1 admin admin] PATH_INFO: `/v3/projects/febcab00c1c14107b16651b1d98e168e/users/b8e024b5c94a4cacb1814cd055d3c29a/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.751344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cd3f9-bf2b-41e1-9cdc-76afdee2e3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febcab00c1c14107b16651b1d98e168e, user_id=b8e024b5c94a4cacb1814cd055d3c29a, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:18.755488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cd3f9-bf2b-41e1-9cdc-76afdee2e3f1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:18.777394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1019/2041] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:18 2022] PUT /identity/v3/projects/febcab00c1c14107b16651b1d98e168e/users/b8e024b5c94a4cacb1814cd055d3c29a/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:18:18.792120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5ddc-3829-473d-825b-7cc9ce9078cb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:18.792509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5ddc-3829-473d-825b-7cc9ce9078cb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:18.792862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a5ddc-3829-473d-825b-7cc9ce9078cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:18.861975 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-831a5ddc-3829-473d-825b-7cc9ce9078cb None None] MFA Rules not processed for user `b8e024b5c94a4cacb1814cd055d3c29a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:18.863247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-831a5ddc-3829-473d-825b-7cc9ce9078cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:18.948217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1023/2042] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:18.952707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cd3f9-bf2b-41e1-9cdc-76afdee2e3f1 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:19.045689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cd3f9-bf2b-41e1-9cdc-76afdee2e3f1 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:19.051011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31646-00e5-4e10-8a0b-7e8bbdcfb737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:19.052726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31646-00e5-4e10-8a0b-7e8bbdcfb737 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.053849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31646-00e5-4e10-8a0b-7e8bbdcfb737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.054130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b31646-00e5-4e10-8a0b-7e8bbdcfb737 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.054783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b31646-00e5-4e10-8a0b-7e8bbdcfb737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:19.057763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b31646-00e5-4e10-8a0b-7e8bbdcfb737 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:19.095946 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1020/2043] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:18:18 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:19.101923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5ddc-3829-473d-825b-7cc9ce9078cb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:19.113192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a5ddc-3829-473d-825b-7cc9ce9078cb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:19.205377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddaa15-e671-4973-a5e3-97d2b88d4d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:19.206574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddaa15-e671-4973-a5e3-97d2b88d4d62 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.206574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddaa15-e671-4973-a5e3-97d2b88d4d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.206820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddaa15-e671-4973-a5e3-97d2b88d4d62 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.207514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddaa15-e671-4973-a5e3-97d2b88d4d62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:19.209742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddaa15-e671-4973-a5e3-97d2b88d4d62 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:19.243527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1024/2044] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:18:19 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:18:19.244818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31646-00e5-4e10-8a0b-7e8bbdcfb737 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:19.260439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b31646-00e5-4e10-8a0b-7e8bbdcfb737 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:19.269005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523abf73-c4b6-4e7a-ae9b-e267829b45ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:19.269636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523abf73-c4b6-4e7a-ae9b-e267829b45ad admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.269916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523abf73-c4b6-4e7a-ae9b-e267829b45ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.270385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523abf73-c4b6-4e7a-ae9b-e267829b45ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.271301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523abf73-c4b6-4e7a-ae9b-e267829b45ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:19.273071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523abf73-c4b6-4e7a-ae9b-e267829b45ad admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:19.291090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1021/2045] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:19.294832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddaa15-e671-4973-a5e3-97d2b88d4d62 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:19.309499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddaa15-e671-4973-a5e3-97d2b88d4d62 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:19.314631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a665a-68c5-4eca-862f-436f81d45f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:19.315853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a665a-68c5-4eca-862f-436f81d45f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.316140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a665a-68c5-4eca-862f-436f81d45f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.316425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a665a-68c5-4eca-862f-436f81d45f9d admin admin] PATH_INFO: `/v3/projects/021580eb1fb948c7bd1dbc6ad8430d63/users/8dcea0bb84ff42d185c88d4e9573b269/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.320239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:19.340518 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a665a-68c5-4eca-862f-436f81d45f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=021580eb1fb948c7bd1dbc6ad8430d63, user_id=8dcea0bb84ff42d185c88d4e9573b269, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:19.344159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a665a-68c5-4eca-862f-436f81d45f9d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:19.368648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1025/2046] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:19 2022] PUT /identity/v3/projects/021580eb1fb948c7bd1dbc6ad8430d63/users/8dcea0bb84ff42d185c88d4e9573b269/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:19.369791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523abf73-c4b6-4e7a-ae9b-e267829b45ad admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:19.389025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523abf73-c4b6-4e7a-ae9b-e267829b45ad admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:19.490684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:19.500686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced95602-26ad-45df-bf8c-41466485032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:19.501954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced95602-26ad-45df-bf8c-41466485032d admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.502461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced95602-26ad-45df-bf8c-41466485032d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.502960 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced95602-26ad-45df-bf8c-41466485032d admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.506316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced95602-26ad-45df-bf8c-41466485032d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:19.508612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced95602-26ad-45df-bf8c-41466485032d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:19.528300 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1022/2047] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:18:19.529979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a665a-68c5-4eca-862f-436f81d45f9d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:19.545129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a665a-68c5-4eca-862f-436f81d45f9d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:19.550874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:19.551973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.552477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.552986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] PATH_INFO: `/v3/projects/021580eb1fb948c7bd1dbc6ad8430d63/users/8dcea0bb84ff42d185c88d4e9573b269/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.555316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=021580eb1fb948c7bd1dbc6ad8430d63, user_id=8dcea0bb84ff42d185c88d4e9573b269, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:19.558375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:19.573861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1026/2048] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:19 2022] PUT /identity/v3/projects/021580eb1fb948c7bd1dbc6ad8430d63/users/8dcea0bb84ff42d185c88d4e9573b269/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:19.586065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06c68-90ca-4ab0-85e9-9ab0d689c211 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.586400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06c68-90ca-4ab0-85e9-9ab0d689c211 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.588732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f06c68-90ca-4ab0-85e9-9ab0d689c211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.620590 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-22f06c68-90ca-4ab0-85e9-9ab0d689c211 None None] MFA Rules not processed for user `8dcea0bb84ff42d185c88d4e9573b269`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:19.621727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-22f06c68-90ca-4ab0-85e9-9ab0d689c211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:19.753661 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1023/2049] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:19.765168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:19.767221 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:19.890072 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d432704-2d1b-4bbf-a8e8-e7580adec3b5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:19.895384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6f41b6-d0fb-44d8-b696-fc5ae6818451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:19.896782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f41b6-d0fb-44d8-b696-fc5ae6818451 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:19.897185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f41b6-d0fb-44d8-b696-fc5ae6818451 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:19.897479 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6f41b6-d0fb-44d8-b696-fc5ae6818451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:19.911351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f41b6-d0fb-44d8-b696-fc5ae6818451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:19.916861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6f41b6-d0fb-44d8-b696-fc5ae6818451 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:19.934878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1027/2050] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:19 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:22.397230 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f06c68-90ca-4ab0-85e9-9ab0d689c211 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:22.555458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f06c68-90ca-4ab0-85e9-9ab0d689c211 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:22.572091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:22.572786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:22.573070 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:22.573353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] PATH_INFO: `/v3/users/b8e024b5c94a4cacb1814cd055d3c29a` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:22.574685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8e024b5c94a4cacb1814cd055d3c29a)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:22.576628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:22.679333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8e024b5c94a4cacb1814cd055d3c29a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:22.713872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8e024b5c94a4cacb1814cd055d3c29a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:22.715375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8e024b5c94a4cacb1814cd055d3c29a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:22.740300 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8e024b5c94a4cacb1814cd055d3c29a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:22.752225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8e024b5c94a4cacb1814cd055d3c29a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:22.757147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:22.768950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1024/2051] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:22 2022] DELETE /identity/v3/users/b8e024b5c94a4cacb1814cd055d3c29a => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:22.772139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e7b1-6954-4838-b613-51c6fe0c5aaa None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:22.772139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e7b1-6954-4838-b613-51c6fe0c5aaa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:22.773130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22e7b1-6954-4838-b613-51c6fe0c5aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:22.836890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ca22e7b1-6954-4838-b613-51c6fe0c5aaa None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:22.843691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ca22e7b1-6954-4838-b613-51c6fe0c5aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:22.996198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1028/2052] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:23.005078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:23.008140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:23.129148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df8ff6-b7b6-4e20-a225-1c42fb3ea5d0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:23.133492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da732c-1a98-4add-bf02-5b4d2227a828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:23.138358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da732c-1a98-4add-bf02-5b4d2227a828 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:23.138652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da732c-1a98-4add-bf02-5b4d2227a828 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:23.138948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da732c-1a98-4add-bf02-5b4d2227a828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:23.167243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da732c-1a98-4add-bf02-5b4d2227a828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:23.167243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7da732c-1a98-4add-bf02-5b4d2227a828 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:23.201663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1025/2053] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:23.902807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e7b1-6954-4838-b613-51c6fe0c5aaa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:23.904477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ca22e7b1-6954-4838-b613-51c6fe0c5aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:23.932085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22e7b1-6954-4838-b613-51c6fe0c5aaa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:23.940754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:23.942945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:23.943639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:23.944660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] PATH_INFO: `/v3/projects/febcab00c1c14107b16651b1d98e168e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:23.946962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=febcab00c1c14107b16651b1d98e168e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:23.949768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:24.075754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febcab00c1c14107b16651b1d98e168e', 'request_id': 'req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.094510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febcab00c1c14107b16651b1d98e168e', 'request_id': 'req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.115527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febcab00c1c14107b16651b1d98e168e', 'request_id': 'req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.132275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:24.144805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] The token cache is being invalidate because project febcab00c1c14107b16651b1d98e168e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:24.145259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.149826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.151867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1029/2054] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:23 2022] DELETE /identity/v3/projects/febcab00c1c14107b16651b1d98e168e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:24.157953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da732c-1a98-4add-bf02-5b4d2227a828 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:24.179186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da732c-1a98-4add-bf02-5b4d2227a828 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:24.272616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:24.273546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:24.274015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:24.274439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] PATH_INFO: `/v3/users/8dcea0bb84ff42d185c88d4e9573b269` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:24.276348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dcea0bb84ff42d185c88d4e9573b269)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:24.278436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:24.416355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dcea0bb84ff42d185c88d4e9573b269'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.455592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dcea0bb84ff42d185c88d4e9573b269'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.458423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dcea0bb84ff42d185c88d4e9573b269'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.470734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dcea0bb84ff42d185c88d4e9573b269'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.495469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dcea0bb84ff42d185c88d4e9573b269'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:24.501714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1026/2055] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:24 2022] DELETE /identity/v3/users/8dcea0bb84ff42d185c88d4e9573b269 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:25.109487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:25.112760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:25.186602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:25.188870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:25.245393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:25.305525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffaaa1-69f2-4584-b4e6-e65e32c49fb5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:25.336217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5fdfd-74bf-4952-bfc3-cc29d43096f0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:25.344573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:25.345607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.346023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.347452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] PATH_INFO: `/v3/projects/021580eb1fb948c7bd1dbc6ad8430d63` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:25.350941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=021580eb1fb948c7bd1dbc6ad8430d63)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:25.357781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:25.461312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8164b914-314f-4d61-9775-9ff8fe861b75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:25.464379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164b914-314f-4d61-9775-9ff8fe861b75 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.464884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164b914-314f-4d61-9775-9ff8fe861b75 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.466278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8164b914-314f-4d61-9775-9ff8fe861b75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:25.471803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8164b914-314f-4d61-9775-9ff8fe861b75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:25.491607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '021580eb1fb948c7bd1dbc6ad8430d63', 'request_id': 'req-b7b54583-442d-4c4f-b503-754bbd87cac9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:25.501471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164b914-314f-4d61-9775-9ff8fe861b75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:25.503267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8164b914-314f-4d61-9775-9ff8fe861b75 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:25.506853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '021580eb1fb948c7bd1dbc6ad8430d63', 'request_id': 'req-b7b54583-442d-4c4f-b503-754bbd87cac9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:25.518414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1030/2056] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:25.532711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '021580eb1fb948c7bd1dbc6ad8430d63', 'request_id': 'req-b7b54583-442d-4c4f-b503-754bbd87cac9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:25.563944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] The token cache is being invalidate because project 021580eb1fb948c7bd1dbc6ad8430d63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:25.565356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:25.566267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b54583-442d-4c4f-b503-754bbd87cac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:25.582977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1027/2057] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:25 2022] DELETE /identity/v3/projects/021580eb1fb948c7bd1dbc6ad8430d63 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:25.587837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2f84-f2a3-45c3-beec-e87e139ea216 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.588151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2f84-f2a3-45c3-beec-e87e139ea216 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.588386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda2f84-f2a3-45c3-beec-e87e139ea216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:25.615526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6dda2f84-f2a3-45c3-beec-e87e139ea216 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:25.617084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6dda2f84-f2a3-45c3-beec-e87e139ea216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:25.736660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1031/2058] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:25.738328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec89813-22e6-4cb8-95e3-f94c380ff170 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.738621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec89813-22e6-4cb8-95e3-f94c380ff170 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.738911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec89813-22e6-4cb8-95e3-f94c380ff170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:25.772516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1ec89813-22e6-4cb8-95e3-f94c380ff170 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:25.775088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1ec89813-22e6-4cb8-95e3-f94c380ff170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:25.788493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1028/2059] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:25.789331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2f84-f2a3-45c3-beec-e87e139ea216 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:25.800275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda2f84-f2a3-45c3-beec-e87e139ea216 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:25.805814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:25.806687 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.806976 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.807283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:25.807948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:18:25.808253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:25.809870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:25.824440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1032/2060] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:18:25.828456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec89813-22e6-4cb8-95e3-f94c380ff170 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:25.838581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec89813-22e6-4cb8-95e3-f94c380ff170 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:25.843853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34974049-7a51-459a-a6e0-58877472d8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:25.844466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34974049-7a51-459a-a6e0-58877472d8af admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.844746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34974049-7a51-459a-a6e0-58877472d8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.845017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34974049-7a51-459a-a6e0-58877472d8af admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:25.845664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34974049-7a51-459a-a6e0-58877472d8af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:25.847776 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34974049-7a51-459a-a6e0-58877472d8af admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:25.888132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1029/2061] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:18:25 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:18:25.893642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:25.904558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83d94c-93ff-430d-a0b7-3139f830c465 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:25.922466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:25.995410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf84264-7eb1-48cd-9e23-dc9afc88933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:25.997156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf84264-7eb1-48cd-9e23-dc9afc88933e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:25.998226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf84264-7eb1-48cd-9e23-dc9afc88933e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:25.999084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf84264-7eb1-48cd-9e23-dc9afc88933e admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.000276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf84264-7eb1-48cd-9e23-dc9afc88933e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.003451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf84264-7eb1-48cd-9e23-dc9afc88933e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.046693 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34974049-7a51-459a-a6e0-58877472d8af admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.055537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1033/2062] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:18:25 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:26.064614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34974049-7a51-459a-a6e0-58877472d8af admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.077005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0b3bd-cbaa-4e6a-93aa-f5ba16b3b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.077651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b3bd-cbaa-4e6a-93aa-f5ba16b3b728 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.077937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b3bd-cbaa-4e6a-93aa-f5ba16b3b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.078216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b3bd-cbaa-4e6a-93aa-f5ba16b3b728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.080535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0b3bd-cbaa-4e6a-93aa-f5ba16b3b728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.082191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0b3bd-cbaa-4e6a-93aa-f5ba16b3b728 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.111696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf84264-7eb1-48cd-9e23-dc9afc88933e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.114948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1030/2063] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:26.124398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf84264-7eb1-48cd-9e23-dc9afc88933e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.130889 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd26218-b9b9-45db-aea8-af745350504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.133328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26218-b9b9-45db-aea8-af745350504d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.133328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26218-b9b9-45db-aea8-af745350504d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.133328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd26218-b9b9-45db-aea8-af745350504d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.133630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd26218-b9b9-45db-aea8-af745350504d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.137284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd26218-b9b9-45db-aea8-af745350504d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.154335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1034/2064] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:18:26.155736 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0b3bd-cbaa-4e6a-93aa-f5ba16b3b728 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.168212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0b3bd-cbaa-4e6a-93aa-f5ba16b3b728 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.173528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8f34c-61e4-4abe-8e6f-ba83a026a051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.174472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8f34c-61e4-4abe-8e6f-ba83a026a051 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.175422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8f34c-61e4-4abe-8e6f-ba83a026a051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.175966 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8f34c-61e4-4abe-8e6f-ba83a026a051 admin admin] PATH_INFO: `/v3/projects/a5c05853c82f4df58508c76eeafb4754/users/4950884124cb4eadb86d8d1e042a318b/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.197130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8f34c-61e4-4abe-8e6f-ba83a026a051 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c05853c82f4df58508c76eeafb4754, user_id=4950884124cb4eadb86d8d1e042a318b, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.200442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8f34c-61e4-4abe-8e6f-ba83a026a051 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.217246 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1031/2065] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:26 2022] PUT /identity/v3/projects/a5c05853c82f4df58508c76eeafb4754/users/4950884124cb4eadb86d8d1e042a318b/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:18:26.229194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3401-4a71-4513-ac5f-65d8608590a0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.229876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3401-4a71-4513-ac5f-65d8608590a0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.230346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c3401-4a71-4513-ac5f-65d8608590a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.277704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fa5c3401-4a71-4513-ac5f-65d8608590a0 None None] MFA Rules not processed for user `4950884124cb4eadb86d8d1e042a318b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:26.279437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fa5c3401-4a71-4513-ac5f-65d8608590a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:26.372655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1035/2066] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:26.377974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8f34c-61e4-4abe-8e6f-ba83a026a051 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.455455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:26.478478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8f34c-61e4-4abe-8e6f-ba83a026a051 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.485629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4952-7fe0-4177-b2a5-2d492577ff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.486259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4952-7fe0-4177-b2a5-2d492577ff76 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.486611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4952-7fe0-4177-b2a5-2d492577ff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.486947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4952-7fe0-4177-b2a5-2d492577ff76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.487900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda4952-7fe0-4177-b2a5-2d492577ff76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.490653 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eda4952-7fe0-4177-b2a5-2d492577ff76 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.533411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1032/2067] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:18:26 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:26.537540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c3401-4a71-4513-ac5f-65d8608590a0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.548657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c3401-4a71-4513-ac5f-65d8608590a0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.615882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9fd79-a8eb-47d1-a3f3-e34b95f94df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.616494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9fd79-a8eb-47d1-a3f3-e34b95f94df6 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.616780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9fd79-a8eb-47d1-a3f3-e34b95f94df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.617059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9fd79-a8eb-47d1-a3f3-e34b95f94df6 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.617715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9fd79-a8eb-47d1-a3f3-e34b95f94df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.621373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e9fd79-a8eb-47d1-a3f3-e34b95f94df6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.660795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1036/2068] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:18:26 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:26.666019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4952-7fe0-4177-b2a5-2d492577ff76 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.679393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4952-7fe0-4177-b2a5-2d492577ff76 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.687041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392636a-402e-4186-8da3-e42e635cb779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.688232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392636a-402e-4186-8da3-e42e635cb779 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.688752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392636a-402e-4186-8da3-e42e635cb779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.689263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c392636a-402e-4186-8da3-e42e635cb779 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.690130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392636a-402e-4186-8da3-e42e635cb779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.692112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c392636a-402e-4186-8da3-e42e635cb779 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.714606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1033/2069] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:26.715958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9fd79-a8eb-47d1-a3f3-e34b95f94df6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.727819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9fd79-a8eb-47d1-a3f3-e34b95f94df6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.736704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41495781-32bd-446d-aa7e-d8b27196bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.737388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41495781-32bd-446d-aa7e-d8b27196bc70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.737672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41495781-32bd-446d-aa7e-d8b27196bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.737948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41495781-32bd-446d-aa7e-d8b27196bc70 admin admin] PATH_INFO: `/v3/projects/f1045c9b793b46e384d7cf057cca299a/users/a2dfac0ff68144faa2d863623ced2ecf/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.753940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:26.762190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41495781-32bd-446d-aa7e-d8b27196bc70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1045c9b793b46e384d7cf057cca299a, user_id=a2dfac0ff68144faa2d863623ced2ecf, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.766326 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41495781-32bd-446d-aa7e-d8b27196bc70 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.784222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1037/2070] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:26 2022] PUT /identity/v3/projects/f1045c9b793b46e384d7cf057cca299a/users/a2dfac0ff68144faa2d863623ced2ecf/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:26.790452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392636a-402e-4186-8da3-e42e635cb779 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.803991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c392636a-402e-4186-8da3-e42e635cb779 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.883425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68cb016-2349-4bf8-bd38-b3068009d56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.884058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb016-2349-4bf8-bd38-b3068009d56b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.884347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb016-2349-4bf8-bd38-b3068009d56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.886701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68cb016-2349-4bf8-bd38-b3068009d56b admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.887486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb016-2349-4bf8-bd38-b3068009d56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.889176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68cb016-2349-4bf8-bd38-b3068009d56b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.908267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1034/2071] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:26.909023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41495781-32bd-446d-aa7e-d8b27196bc70 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:26.928131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41495781-32bd-446d-aa7e-d8b27196bc70 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:26.936505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:26.937454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.937924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.938436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] PATH_INFO: `/v3/projects/f1045c9b793b46e384d7cf057cca299a/users/a2dfac0ff68144faa2d863623ced2ecf/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:26.944627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1045c9b793b46e384d7cf057cca299a, user_id=a2dfac0ff68144faa2d863623ced2ecf, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:26.947716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:26.979398 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1038/2072] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:26 2022] PUT /identity/v3/projects/f1045c9b793b46e384d7cf057cca299a/users/a2dfac0ff68144faa2d863623ced2ecf/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:18:26.990599 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121cc49-bc57-4330-8d8d-acbe4f8700f6 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:26.990886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121cc49-bc57-4330-8d8d-acbe4f8700f6 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:26.991222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121cc49-bc57-4330-8d8d-acbe4f8700f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:27.028440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-2121cc49-bc57-4330-8d8d-acbe4f8700f6 None None] MFA Rules not processed for user `a2dfac0ff68144faa2d863623ced2ecf`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:27.029921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2121cc49-bc57-4330-8d8d-acbe4f8700f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:27.171328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1035/2073] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:27.200550 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:27.202795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:27.291025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993517d-3256-4369-90b3-71d526772b26 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:27.296395 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:27.298281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:27.298722 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:27.299195 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:27.315776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:27.319947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:27.340740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1039/2074] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:27.478289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121cc49-bc57-4330-8d8d-acbe4f8700f6 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:27.505213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121cc49-bc57-4330-8d8d-acbe4f8700f6 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:27.517049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82128038-57d1-4ef5-8a7c-3148d69a1af5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:27.517691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82128038-57d1-4ef5-8a7c-3148d69a1af5 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:27.517976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82128038-57d1-4ef5-8a7c-3148d69a1af5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:27.518254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82128038-57d1-4ef5-8a7c-3148d69a1af5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:27.610510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:27.662388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82128038-57d1-4ef5-8a7c-3148d69a1af5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:27.663986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82128038-57d1-4ef5-8a7c-3148d69a1af5 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:27.684362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1036/2075] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:29.883846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:29.889861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:30.006784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c05bc-b573-48ed-adc6-183d1788e1f8 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:30.012208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:30.013035 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:30.013590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:30.014028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] PATH_INFO: `/v3/users/a769bcc0330a4171948a3cc930bd1ee2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:30.015511 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a769bcc0330a4171948a3cc930bd1ee2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:30.018078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:30.111734 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a769bcc0330a4171948a3cc930bd1ee2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.131190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a769bcc0330a4171948a3cc930bd1ee2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.132241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a769bcc0330a4171948a3cc930bd1ee2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.142667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a769bcc0330a4171948a3cc930bd1ee2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.142667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a769bcc0330a4171948a3cc930bd1ee2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.156520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:30.167194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1040/2076] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:29 2022] DELETE /identity/v3/users/a769bcc0330a4171948a3cc930bd1ee2 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:18:30.418243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82128038-57d1-4ef5-8a7c-3148d69a1af5 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:30.434335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82128038-57d1-4ef5-8a7c-3148d69a1af5 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:30.533392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:30.534276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:30.534557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:30.534825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] PATH_INFO: `/v3/projects/e6ab0fbf44394afda8dd8f49818ccc63` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:30.536852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ab0fbf44394afda8dd8f49818ccc63)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:30.538986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:30.593965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ab0fbf44394afda8dd8f49818ccc63', 'request_id': 'req-96717d38-fd0c-4981-b156-09d60260fcef'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.600699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ab0fbf44394afda8dd8f49818ccc63', 'request_id': 'req-96717d38-fd0c-4981-b156-09d60260fcef'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.609447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ab0fbf44394afda8dd8f49818ccc63', 'request_id': 'req-96717d38-fd0c-4981-b156-09d60260fcef'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.621767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] The token cache is being invalidate because project e6ab0fbf44394afda8dd8f49818ccc63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:30.622078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.622748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.624471 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1037/2077] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:30 2022] DELETE /identity/v3/projects/e6ab0fbf44394afda8dd8f49818ccc63 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:30.628295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:30.629883 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:30.645775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69087610-94e8-4d6c-85e0-af87a77b454f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:30.713522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:30.714267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:30.714630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:30.714941 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] PATH_INFO: `/v3/users/8e6556d4eaa44d66a52fe63a521523d4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:30.716160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e6556d4eaa44d66a52fe63a521523d4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:30.718082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:30.778413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e6556d4eaa44d66a52fe63a521523d4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.794079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e6556d4eaa44d66a52fe63a521523d4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.795035 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e6556d4eaa44d66a52fe63a521523d4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.809612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e6556d4eaa44d66a52fe63a521523d4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.810902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e6556d4eaa44d66a52fe63a521523d4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:30.829986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1041/2078] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:30 2022] DELETE /identity/v3/users/8e6556d4eaa44d66a52fe63a521523d4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:31.071521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:31.072977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:31.088680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96717d38-fd0c-4981-b156-09d60260fcef admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:31.198109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:31.199353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:31.199911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:31.200267 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] PATH_INFO: `/v3/projects/aa714009ed494760aba72453db740147` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:31.202557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa714009ed494760aba72453db740147)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:31.205273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:31.219468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:31.297159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa714009ed494760aba72453db740147', 'request_id': 'req-ce415a4e-4d23-4de5-a122-f2d27599c9a0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:31.304880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa714009ed494760aba72453db740147', 'request_id': 'req-ce415a4e-4d23-4de5-a122-f2d27599c9a0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:31.314582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa714009ed494760aba72453db740147', 'request_id': 'req-ce415a4e-4d23-4de5-a122-f2d27599c9a0'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:31.331920 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] The token cache is being invalidate because project aa714009ed494760aba72453db740147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:31.332315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:31.333084 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ce415a4e-4d23-4de5-a122-f2d27599c9a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:31.336504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1038/2079] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:31 2022] DELETE /identity/v3/projects/aa714009ed494760aba72453db740147 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:18:33.448812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:33.451549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:33.464946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7610f-a80d-4934-a931-1c4cf4ab177f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:33.530044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:33.530807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:33.531249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:33.531887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] PATH_INFO: `/v3/users/4950884124cb4eadb86d8d1e042a318b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:33.532895 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4950884124cb4eadb86d8d1e042a318b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:33.534924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:33.565964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:33.617872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4950884124cb4eadb86d8d1e042a318b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:33.630748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4950884124cb4eadb86d8d1e042a318b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:33.631582 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4950884124cb4eadb86d8d1e042a318b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:33.640975 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4950884124cb4eadb86d8d1e042a318b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:33.641598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4950884124cb4eadb86d8d1e042a318b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:33.655729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1042/2080] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:33 2022] DELETE /identity/v3/users/4950884124cb4eadb86d8d1e042a318b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:33.660263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a3399-6b54-4f08-aa20-b7356ae1ba01 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:33.661284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a3399-6b54-4f08-aa20-b7356ae1ba01 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:33.662272 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a3399-6b54-4f08-aa20-b7356ae1ba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:33.682787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-0c7a3399-6b54-4f08-aa20-b7356ae1ba01 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:33.684043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0c7a3399-6b54-4f08-aa20-b7356ae1ba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:33.742430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1039/2081] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:33.752201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:33.753633 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:33.813073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a8a32-38d0-440c-8c6e-1d40b15b7268 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:33.816790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb7752-7862-45d6-9e85-83d98d39270c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:33.817349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb7752-7862-45d6-9e85-83d98d39270c service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:33.817744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb7752-7862-45d6-9e85-83d98d39270c service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:33.818031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb7752-7862-45d6-9e85-83d98d39270c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:33.827023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb7752-7862-45d6-9e85-83d98d39270c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:33.828480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb7752-7862-45d6-9e85-83d98d39270c service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:33.838639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1043/2082] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:34.261051 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a3399-6b54-4f08-aa20-b7356ae1ba01 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:34.262759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0c7a3399-6b54-4f08-aa20-b7356ae1ba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:34.275599 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a3399-6b54-4f08-aa20-b7356ae1ba01 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:34.281398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:34.281556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:34.281949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:34.282288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] PATH_INFO: `/v3/projects/a5c05853c82f4df58508c76eeafb4754` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:34.283426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c05853c82f4df58508c76eeafb4754)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:34.285384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:34.340040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c05853c82f4df58508c76eeafb4754', 'request_id': 'req-ddb9826d-50af-4ab2-95db-8cc3e0264590'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.346954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c05853c82f4df58508c76eeafb4754', 'request_id': 'req-ddb9826d-50af-4ab2-95db-8cc3e0264590'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.355536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c05853c82f4df58508c76eeafb4754', 'request_id': 'req-ddb9826d-50af-4ab2-95db-8cc3e0264590'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.367562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] The token cache is being invalidate because project a5c05853c82f4df58508c76eeafb4754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:34.367924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.368557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.370312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1040/2083] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:34 2022] DELETE /identity/v3/projects/a5c05853c82f4df58508c76eeafb4754 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:34.374093 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb7752-7862-45d6-9e85-83d98d39270c service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:34.384631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb7752-7862-45d6-9e85-83d98d39270c service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:34.441948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:34.443140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:34.443423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:34.443700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] PATH_INFO: `/v3/users/a2dfac0ff68144faa2d863623ced2ecf` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:34.444950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2dfac0ff68144faa2d863623ced2ecf)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:34.446937 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:34.492945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:34.495652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2dfac0ff68144faa2d863623ced2ecf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.509799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2dfac0ff68144faa2d863623ced2ecf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.511781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2dfac0ff68144faa2d863623ced2ecf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.521227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2dfac0ff68144faa2d863623ced2ecf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.521888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a98d5ce5-d057-4b72-ba84-130a463ce415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2dfac0ff68144faa2d863623ced2ecf'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.535281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1044/2084] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:34 2022] DELETE /identity/v3/users/a2dfac0ff68144faa2d863623ced2ecf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:34.819746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:34.821395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:34.839494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb9826d-50af-4ab2-95db-8cc3e0264590 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:34.864445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:34.905235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:34.905905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:34.906195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:34.906485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] PATH_INFO: `/v3/projects/f1045c9b793b46e384d7cf057cca299a` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:34.907698 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1045c9b793b46e384d7cf057cca299a)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:34.909669 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:34.957795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1045c9b793b46e384d7cf057cca299a', 'request_id': 'req-ccd6bb93-817f-4bac-9623-26ccb6a68981'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.963818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1045c9b793b46e384d7cf057cca299a', 'request_id': 'req-ccd6bb93-817f-4bac-9623-26ccb6a68981'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.970972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1045c9b793b46e384d7cf057cca299a', 'request_id': 'req-ccd6bb93-817f-4bac-9623-26ccb6a68981'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.982083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] The token cache is being invalidate because project f1045c9b793b46e384d7cf057cca299a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:34.982419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.983044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ccd6bb93-817f-4bac-9623-26ccb6a68981 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:34.984633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1041/2085] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:34 2022] DELETE /identity/v3/projects/f1045c9b793b46e384d7cf057cca299a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:34.989541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f57aa6-9b2c-4a2f-a2b0-bac89639a9d3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:34.989833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f57aa6-9b2c-4a2f-a2b0-bac89639a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:34.990141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f57aa6-9b2c-4a2f-a2b0-bac89639a9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.010828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-63f57aa6-9b2c-4a2f-a2b0-bac89639a9d3 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:35.012062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-63f57aa6-9b2c-4a2f-a2b0-bac89639a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:35.071108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1045/2086] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:35.078924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc9aa2-b00d-4702-a508-412d304ea1b5 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.079265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc9aa2-b00d-4702-a508-412d304ea1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.079572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc9aa2-b00d-4702-a508-412d304ea1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.100366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-51cc9aa2-b00d-4702-a508-412d304ea1b5 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:35.101558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-51cc9aa2-b00d-4702-a508-412d304ea1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:35.109711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1042/2087] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:35.114389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f57aa6-9b2c-4a2f-a2b0-bac89639a9d3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:35.124126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f57aa6-9b2c-4a2f-a2b0-bac89639a9d3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:35.128672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:35.129263 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.129576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.130036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.130690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:18:35.130999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:35.132629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:35.144580 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1046/2088] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:18:35.148889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc9aa2-b00d-4702-a508-412d304ea1b5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:35.158429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc9aa2-b00d-4702-a508-412d304ea1b5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:35.163747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaee4c-4335-4106-afeb-77907a29a031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:35.164381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaee4c-4335-4106-afeb-77907a29a031 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.164690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaee4c-4335-4106-afeb-77907a29a031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.164971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aaee4c-4335-4106-afeb-77907a29a031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.165622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaee4c-4335-4106-afeb-77907a29a031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:35.167766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aaee4c-4335-4106-afeb-77907a29a031 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:35.195692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1043/2089] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:18:35 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:35.199821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:35.210370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b86655-686b-4308-9c4a-fbcc24ccfc62 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:35.267750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a5d3c-a3f5-44ad-b93d-472c0a196105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:35.268407 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a5d3c-a3f5-44ad-b93d-472c0a196105 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.268719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a5d3c-a3f5-44ad-b93d-472c0a196105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.269059 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a5d3c-a3f5-44ad-b93d-472c0a196105 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.269738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a5d3c-a3f5-44ad-b93d-472c0a196105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:35.272537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a5d3c-a3f5-44ad-b93d-472c0a196105 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:35.298438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1047/2090] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:18:35 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:35.299702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaee4c-4335-4106-afeb-77907a29a031 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:35.309940 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aaee4c-4335-4106-afeb-77907a29a031 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:35.314445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12082863-bddd-4f0f-a3b6-e68dfa89c936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:35.315083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12082863-bddd-4f0f-a3b6-e68dfa89c936 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.315370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12082863-bddd-4f0f-a3b6-e68dfa89c936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.315644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12082863-bddd-4f0f-a3b6-e68dfa89c936 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.316284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12082863-bddd-4f0f-a3b6-e68dfa89c936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:35.317982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12082863-bddd-4f0f-a3b6-e68dfa89c936 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:35.327838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1044/2091] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:18:35.332202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a5d3c-a3f5-44ad-b93d-472c0a196105 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:35.342948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a5d3c-a3f5-44ad-b93d-472c0a196105 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:35.347591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a956f-30de-484b-b5e0-3e45e03ac525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:35.348806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a956f-30de-484b-b5e0-3e45e03ac525 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.349098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a956f-30de-484b-b5e0-3e45e03ac525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.349379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a956f-30de-484b-b5e0-3e45e03ac525 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.350082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a956f-30de-484b-b5e0-3e45e03ac525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:35.351875 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a956f-30de-484b-b5e0-3e45e03ac525 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:35.362885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1048/2092] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:35.367496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12082863-bddd-4f0f-a3b6-e68dfa89c936 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:35.377431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12082863-bddd-4f0f-a3b6-e68dfa89c936 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:35.381951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:35.382588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.382866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.383197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] PATH_INFO: `/v3/projects/0c821fed4e884734bc8a260159334bc8/users/b98a3701f11d4eab921e3b47298c1db7/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.399752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c821fed4e884734bc8a260159334bc8, user_id=b98a3701f11d4eab921e3b47298c1db7, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:35.402560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:35.415213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1045/2093] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:35 2022] PUT /identity/v3/projects/0c821fed4e884734bc8a260159334bc8/users/b98a3701f11d4eab921e3b47298c1db7/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:35.427699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73972-2a17-4df0-9d1f-f32603885e96 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.427982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73972-2a17-4df0-9d1f-f32603885e96 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.428256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b73972-2a17-4df0-9d1f-f32603885e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.457030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-76b73972-2a17-4df0-9d1f-f32603885e96 None None] MFA Rules not processed for user `b98a3701f11d4eab921e3b47298c1db7`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:35.458328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-76b73972-2a17-4df0-9d1f-f32603885e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:35.471499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:35.526631 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1049/2094] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:35.537448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:35.539160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:35.595647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f5b34-3f36-4abd-8d78-dc8a4b10c424 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:35.599888 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b152e-3671-4096-a805-d1aba9b217a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:35.600495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b152e-3671-4096-a805-d1aba9b217a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:35.600782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b152e-3671-4096-a805-d1aba9b217a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:35.601059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b152e-3671-4096-a805-d1aba9b217a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:35.606201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:35.610949 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b152e-3671-4096-a805-d1aba9b217a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:35.612901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b152e-3671-4096-a805-d1aba9b217a1 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:35.623218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1046/2095] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:40.245725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b0adc-dd2b-438d-87ae-331c864282ac None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.246225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b0adc-dd2b-438d-87ae-331c864282ac None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.246327 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b0adc-dd2b-438d-87ae-331c864282ac None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.247814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1050/2096] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:40.255938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704c759-d752-4910-b0c5-4547be62c715 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.256886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704c759-d752-4910-b0c5-4547be62c715 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.257382 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704c759-d752-4910-b0c5-4547be62c715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.281289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a704c759-d752-4910-b0c5-4547be62c715 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:40.282793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a704c759-d752-4910-b0c5-4547be62c715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:40.369228 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1047/2097] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:18:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:40.382974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b0adc-dd2b-438d-87ae-331c864282ac None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:40.384863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b92b0adc-dd2b-438d-87ae-331c864282ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:40.453911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b0adc-dd2b-438d-87ae-331c864282ac None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:40.458086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b9646-5ca6-4850-a294-af4cb036b12e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d0fc260e254d2d80c6927c52cd8a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:40.458720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b9646-5ca6-4850-a294-af4cb036b12e service glance] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.459198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b9646-5ca6-4850-a294-af4cb036b12e service glance] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.459525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b9646-5ca6-4850-a294-af4cb036b12e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.471124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b9646-5ca6-4850-a294-af4cb036b12e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:40.472695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b9646-5ca6-4850-a294-af4cb036b12e service glance] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:40.484557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1051/2098] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:40.557313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641cec2-3a57-43a5-9bb1-4c14485dded8 None None] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.557603 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641cec2-3a57-43a5-9bb1-4c14485dded8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.557879 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4641cec2-3a57-43a5-9bb1-4c14485dded8 None None] PATH_INFO: `/` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.559806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1048/2099] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:40.563944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d21a6-46af-465d-a70c-422e5257ad48 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.564308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d21a6-46af-465d-a70c-422e5257ad48 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.564652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d21a6-46af-465d-a70c-422e5257ad48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.585960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c40d21a6-46af-465d-a70c-422e5257ad48 None None] MFA Rules not processed for user `9c414b04fe0544e3a938661bc957c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:40.587274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c40d21a6-46af-465d-a70c-422e5257ad48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:40.641224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1052/2100] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:18:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:40.653217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641cec2-3a57-43a5-9bb1-4c14485dded8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:40.654897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4641cec2-3a57-43a5-9bb1-4c14485dded8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:40.668324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4641cec2-3a57-43a5-9bb1-4c14485dded8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:40.672054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf63d2-fda2-4214-b8bf-a8bda3dce34e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:40.672658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf63d2-fda2-4214-b8bf-a8bda3dce34e service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.672937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf63d2-fda2-4214-b8bf-a8bda3dce34e service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.673211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf63d2-fda2-4214-b8bf-a8bda3dce34e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.681092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf63d2-fda2-4214-b8bf-a8bda3dce34e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:40.682519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf63d2-fda2-4214-b8bf-a8bda3dce34e service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:40.691314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1049/2101] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:40.780544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2589-0f04-492a-85da-0badfba7a545 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.780909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2589-0f04-492a-85da-0badfba7a545 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.781260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be2589-0f04-492a-85da-0badfba7a545 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.783075 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1053/2102] 10.222.0.42 () {54 vars in 921 bytes} [Mon Oct 24 13:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 24 13:18:40.789985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c559f-1ab7-4ca3-9792-2346a6439727 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.790423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c559f-1ab7-4ca3-9792-2346a6439727 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.790770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c559f-1ab7-4ca3-9792-2346a6439727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.815880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-266c559f-1ab7-4ca3-9792-2346a6439727 None None] MFA Rules not processed for user `2dff78b5eadc42608afbf2b14dd07e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:40.817661 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-266c559f-1ab7-4ca3-9792-2346a6439727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:40.827724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1050/2103] 10.222.0.42 () {58 vars in 1018 bytes} [Mon Oct 24 13:18:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:18:40.838193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2589-0f04-492a-85da-0badfba7a545 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:40.849947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be2589-0f04-492a-85da-0badfba7a545 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:40.854487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:40.855118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:40.856123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:40.856413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:40.869463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:40.870945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:40.882068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1054/2104] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:42.605604 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498cf8-2b8a-4af5-9b8d-35aa9a124521 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:42.606081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498cf8-2b8a-4af5-9b8d-35aa9a124521 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:42.606191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5498cf8-2b8a-4af5-9b8d-35aa9a124521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:42.625038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e5498cf8-2b8a-4af5-9b8d-35aa9a124521 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:42.626269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e5498cf8-2b8a-4af5-9b8d-35aa9a124521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:42.682344 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1051/2105] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:18:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:42.694697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:42.696103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:42.757036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:42.772623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a79cdf4-f0cd-447a-bd46-954d34052dde service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:42.778065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b5359-cf86-474b-95c1-815f7fb202f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:42.778819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b5359-cf86-474b-95c1-815f7fb202f4 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:42.779104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b5359-cf86-474b-95c1-815f7fb202f4 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:42.779383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b5359-cf86-474b-95c1-815f7fb202f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:42.787610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b5359-cf86-474b-95c1-815f7fb202f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:42.789233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b5359-cf86-474b-95c1-815f7fb202f4 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:42.797378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1055/2106] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:18:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:18:43.054007 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5498cf8-2b8a-4af5-9b8d-35aa9a124521 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:43.169922 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5498cf8-2b8a-4af5-9b8d-35aa9a124521 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:43.174824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:43.175497 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:43.175779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:43.176189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] PATH_INFO: `/v3/users/b98a3701f11d4eab921e3b47298c1db7` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:43.177322 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b98a3701f11d4eab921e3b47298c1db7)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:43.179282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:43.203651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:43.245143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b98a3701f11d4eab921e3b47298c1db7'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.260540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b98a3701f11d4eab921e3b47298c1db7'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.261982 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b98a3701f11d4eab921e3b47298c1db7'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.271618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b98a3701f11d4eab921e3b47298c1db7'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.283218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b98a3701f11d4eab921e3b47298c1db7'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.292024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1052/2107] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:18:43 2022] DELETE /identity/v3/users/b98a3701f11d4eab921e3b47298c1db7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:43.297291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39c03-aefa-40f5-b622-63587a5b5fa8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:43.297678 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39c03-aefa-40f5-b622-63587a5b5fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:43.298056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb39c03-aefa-40f5-b622-63587a5b5fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:43.318656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fdb39c03-aefa-40f5-b622-63587a5b5fa8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:43.320553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fdb39c03-aefa-40f5-b622-63587a5b5fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:43.391022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1056/2108] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:43.401722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:43.403263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:43.467729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d314d4-e289-4070-969d-36472162e056 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:43.471273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8867b-9e98-47dc-aa4d-b9841605d958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:43.471883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8867b-9e98-47dc-aa4d-b9841605d958 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:43.472162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8867b-9e98-47dc-aa4d-b9841605d958 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:43.472435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8867b-9e98-47dc-aa4d-b9841605d958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:43.484065 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8867b-9e98-47dc-aa4d-b9841605d958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:43.485711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8867b-9e98-47dc-aa4d-b9841605d958 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:43.497638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1053/2109] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:43.798300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb39c03-aefa-40f5-b622-63587a5b5fa8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:43.802906 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fdb39c03-aefa-40f5-b622-63587a5b5fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:43.815726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb39c03-aefa-40f5-b622-63587a5b5fa8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:43.821923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:43.822661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:43.822950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:43.823342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] PATH_INFO: `/v3/projects/0c821fed4e884734bc8a260159334bc8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:43.824663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c821fed4e884734bc8a260159334bc8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:43.826758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:43.883315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c821fed4e884734bc8a260159334bc8', 'request_id': 'req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.892253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c821fed4e884734bc8a260159334bc8', 'request_id': 'req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.901467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c821fed4e884734bc8a260159334bc8', 'request_id': 'req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.913816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] The token cache is being invalidate because project 0c821fed4e884734bc8a260159334bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:18:43.914108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.914776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86fe8a9b-d63e-4517-a1fc-616f0ad9e2a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:18:43.916301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1057/2110] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:18:43 2022] DELETE /identity/v3/projects/0c821fed4e884734bc8a260159334bc8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:43.921299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d5fd2-a704-4ac5-a7d2-184a782c52ec None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:43.921772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d5fd2-a704-4ac5-a7d2-184a782c52ec None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:43.921896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d5fd2-a704-4ac5-a7d2-184a782c52ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:43.943357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7f2d5fd2-a704-4ac5-a7d2-184a782c52ec None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:43.944588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7f2d5fd2-a704-4ac5-a7d2-184a782c52ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:44.017984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1054/2111] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:44.026324 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e1042-88ed-483a-9b37-09e904758e74 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.026617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e1042-88ed-483a-9b37-09e904758e74 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.026899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e1042-88ed-483a-9b37-09e904758e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.049600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-109e1042-88ed-483a-9b37-09e904758e74 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:44.051615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-109e1042-88ed-483a-9b37-09e904758e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:44.069258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1058/2112] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:44.073485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d5fd2-a704-4ac5-a7d2-184a782c52ec None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:44.084145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d5fd2-a704-4ac5-a7d2-184a782c52ec None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:44.089278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:44.089988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.090451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.090804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.091513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:18:44.091824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:44.093437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:44.114916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1055/2113] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:18:44.119921 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e1042-88ed-483a-9b37-09e904758e74 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:44.131225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e1042-88ed-483a-9b37-09e904758e74 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:44.137698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3264fbb-d5fc-469f-8587-9f333a477eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:44.138611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3264fbb-d5fc-469f-8587-9f333a477eee admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.139092 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3264fbb-d5fc-469f-8587-9f333a477eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.139463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3264fbb-d5fc-469f-8587-9f333a477eee admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.140349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3264fbb-d5fc-469f-8587-9f333a477eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:44.142772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3264fbb-d5fc-469f-8587-9f333a477eee admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:44.180429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1059/2114] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:18:44 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:44.185353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:44.205147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18859672-6fab-4103-bcf6-8b164f5f5f32 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:44.226290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:44.270895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9038ab-163b-4163-9c92-a2cd4811a4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:44.271504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9038ab-163b-4163-9c92-a2cd4811a4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.271795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9038ab-163b-4163-9c92-a2cd4811a4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.272066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9038ab-163b-4163-9c92-a2cd4811a4b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.272705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9038ab-163b-4163-9c92-a2cd4811a4b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:44.274651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9038ab-163b-4163-9c92-a2cd4811a4b7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:44.300169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1056/2115] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:18:44 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:18:44.304534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3264fbb-d5fc-469f-8587-9f333a477eee admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:44.315442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3264fbb-d5fc-469f-8587-9f333a477eee admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:44.321176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdb5f6-0a30-4c1b-9ff5-dacf79c34676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:44.322802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdb5f6-0a30-4c1b-9ff5-dacf79c34676 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.323166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdb5f6-0a30-4c1b-9ff5-dacf79c34676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.323483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdb5f6-0a30-4c1b-9ff5-dacf79c34676 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.324144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdb5f6-0a30-4c1b-9ff5-dacf79c34676 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:44.325869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdb5f6-0a30-4c1b-9ff5-dacf79c34676 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:44.351207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1060/2116] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:44.354814 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9038ab-163b-4163-9c92-a2cd4811a4b7 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:44.366115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9038ab-163b-4163-9c92-a2cd4811a4b7 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:44.370833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190abc29-0f0c-4da5-8f4f-18dc56600457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:44.371478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190abc29-0f0c-4da5-8f4f-18dc56600457 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.371761 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190abc29-0f0c-4da5-8f4f-18dc56600457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.372036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190abc29-0f0c-4da5-8f4f-18dc56600457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.372684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190abc29-0f0c-4da5-8f4f-18dc56600457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:44.374289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190abc29-0f0c-4da5-8f4f-18dc56600457 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:44.386785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1057/2117] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:18:44.391331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdb5f6-0a30-4c1b-9ff5-dacf79c34676 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:44.402863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdb5f6-0a30-4c1b-9ff5-dacf79c34676 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:44.425200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:44.425858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.426378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.426682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] PATH_INFO: `/v3/projects/fda5f4e9a37d4213a970acd70a5be307/users/2112201961554703a62f3c002962fc03/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.445081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda5f4e9a37d4213a970acd70a5be307, user_id=2112201961554703a62f3c002962fc03, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:44.448053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:44.475215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1061/2118] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:18:44 2022] PUT /identity/v3/projects/fda5f4e9a37d4213a970acd70a5be307/users/2112201961554703a62f3c002962fc03/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:18:44.486703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7c058-8485-4bf9-aa69-1d75c81ede6e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.487062 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7c058-8485-4bf9-aa69-1d75c81ede6e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.487368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7c058-8485-4bf9-aa69-1d75c81ede6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.517519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-89f7c058-8485-4bf9-aa69-1d75c81ede6e None None] MFA Rules not processed for user `2112201961554703a62f3c002962fc03`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:18:44.518887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-89f7c058-8485-4bf9-aa69-1d75c81ede6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:44.598295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1058/2119] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:18:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:18:44.610076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:18:44.612052 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:18:44.619362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:44.684854 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc99ac3-31cd-4d44-b1ad-97e2b0346bb4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:18:44.688965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:18:44.689577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:18:44.689864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:18:44.690142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:18:44.699774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:18:44.701277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:18:44.711855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1062/2120] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:18:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:18:54.594345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:18:54.706214 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=24 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:01.919922 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c720-b41b-40bb-a2f8-abd2cfd381e5 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:01.919922 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c720-b41b-40bb-a2f8-abd2cfd381e5 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:01.919922 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c720-b41b-40bb-a2f8-abd2cfd381e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:01.942250 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-ba92c720-b41b-40bb-a2f8-abd2cfd381e5 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:01.943664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ba92c720-b41b-40bb-a2f8-abd2cfd381e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:01.999930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1059/2121] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 24 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:02.013303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:02.014888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:02.095927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39be5db-1e80-4043-9d5b-044b54fa211c service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:02.101733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:02.102343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:02.102628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:02.102909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:02.111394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:02.113575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:02.122118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1063/2122] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:19:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:19:04.647354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92c720-b41b-40bb-a2f8-abd2cfd381e5 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:04.704193 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:04.705793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:04.711479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92c720-b41b-40bb-a2f8-abd2cfd381e5 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:04.717460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:04.718268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:04.718567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:04.718857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] PATH_INFO: `/v3/users/2112201961554703a62f3c002962fc03` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:04.720033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2112201961554703a62f3c002962fc03)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:04.721951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:04.774340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2112201961554703a62f3c002962fc03'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:04.786232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6b316-dc75-456b-9e8d-9c849d248ec7 service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:04.786395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2112201961554703a62f3c002962fc03'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:04.787034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2112201961554703a62f3c002962fc03'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:04.791709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:04.792314 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:04.792624 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:04.792909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:04.794678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2112201961554703a62f3c002962fc03'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:04.794823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:04.802849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-271c4f71-5b6d-492a-a0c6-ff56e31da4bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2112201961554703a62f3c002962fc03'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:04.809219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1060/2123] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:04 2022] DELETE /identity/v3/users/2112201961554703a62f3c002962fc03 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:04.813522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad78d9-ee35-484e-96f2-8c1fc6791ad6 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:04.813809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad78d9-ee35-484e-96f2-8c1fc6791ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:04.814112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad78d9-ee35-484e-96f2-8c1fc6791ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:04.833818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-8fad78d9-ee35-484e-96f2-8c1fc6791ad6 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:04.835088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8fad78d9-ee35-484e-96f2-8c1fc6791ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:04.854908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:04.866136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:04.867843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:04.878112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1064/2124] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:04.900183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1061/2125] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:04.910505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:04.911995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:04.943622 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:04.975103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c7b18-bda5-450a-9e52-85af545aba15 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.025640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b79a6-f9e7-4843-9ce0-a430e3dc0149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.026247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b79a6-f9e7-4843-9ce0-a430e3dc0149 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.026526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b79a6-f9e7-4843-9ce0-a430e3dc0149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.026800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b79a6-f9e7-4843-9ce0-a430e3dc0149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.036747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b79a6-f9e7-4843-9ce0-a430e3dc0149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.038170 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b79a6-f9e7-4843-9ce0-a430e3dc0149 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.047600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1065/2126] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:05.349646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad78d9-ee35-484e-96f2-8c1fc6791ad6 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:05.351079 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8fad78d9-ee35-484e-96f2-8c1fc6791ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:05.364655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad78d9-ee35-484e-96f2-8c1fc6791ad6 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.369881 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.370601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.370884 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.371196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] PATH_INFO: `/v3/projects/fda5f4e9a37d4213a970acd70a5be307` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.372318 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda5f4e9a37d4213a970acd70a5be307)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.374979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.422187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda5f4e9a37d4213a970acd70a5be307', 'request_id': 'req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:05.428327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda5f4e9a37d4213a970acd70a5be307', 'request_id': 'req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:05.435625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda5f4e9a37d4213a970acd70a5be307', 'request_id': 'req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:05.446799 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] The token cache is being invalidate because project fda5f4e9a37d4213a970acd70a5be307 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:05.447116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:05.447769 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ecc1ca26-f634-4d85-ac56-73cf3476ba9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:05.449536 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1062/2127] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:05 2022] DELETE /identity/v3/projects/fda5f4e9a37d4213a970acd70a5be307 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:05.454429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bb90e-c51a-4eff-828d-321336967e1c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.454716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bb90e-c51a-4eff-828d-321336967e1c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.454991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143bb90e-c51a-4eff-828d-321336967e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.476577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-143bb90e-c51a-4eff-828d-321336967e1c None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:05.477785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-143bb90e-c51a-4eff-828d-321336967e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:05.541345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1066/2128] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:05.549129 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f102-0918-4552-a7b4-6bdc1e389be9 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.549441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f102-0918-4552-a7b4-6bdc1e389be9 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.549720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3109f102-0918-4552-a7b4-6bdc1e389be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.571438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-3109f102-0918-4552-a7b4-6bdc1e389be9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:05.572643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3109f102-0918-4552-a7b4-6bdc1e389be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:05.580568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1063/2129] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:05.586066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bb90e-c51a-4eff-828d-321336967e1c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:05.595892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143bb90e-c51a-4eff-828d-321336967e1c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.600810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.601413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.601695 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.601971 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.602623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:05.602929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.604560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.619629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1067/2130] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:19:05.620889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f102-0918-4552-a7b4-6bdc1e389be9 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:05.632008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3109f102-0918-4552-a7b4-6bdc1e389be9 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.637092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f0bdc-7f91-46be-bc0b-8f0d8a64d065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.637699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f0bdc-7f91-46be-bc0b-8f0d8a64d065 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.637979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f0bdc-7f91-46be-bc0b-8f0d8a64d065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.638249 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3f0bdc-7f91-46be-bc0b-8f0d8a64d065 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.638878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f0bdc-7f91-46be-bc0b-8f0d8a64d065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.640825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3f0bdc-7f91-46be-bc0b-8f0d8a64d065 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.673890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1064/2131] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:05 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:05.675113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:05.685423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503a28-fbfe-42e0-b9d0-2be8e84fc1cb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.742115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f7b3-bcd4-42df-ba6c-e37401afefac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.742748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f7b3-bcd4-42df-ba6c-e37401afefac admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.742991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f7b3-bcd4-42df-ba6c-e37401afefac admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.743300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f7b3-bcd4-42df-ba6c-e37401afefac admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.743919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306f7b3-bcd4-42df-ba6c-e37401afefac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.745819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306f7b3-bcd4-42df-ba6c-e37401afefac admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.767153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1068/2132] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:05 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:05.772551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f0bdc-7f91-46be-bc0b-8f0d8a64d065 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:05.782631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3f0bdc-7f91-46be-bc0b-8f0d8a64d065 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.787475 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6a305-b2a0-4fc8-858a-5b1b184229fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.788082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6a305-b2a0-4fc8-858a-5b1b184229fd admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.788365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6a305-b2a0-4fc8-858a-5b1b184229fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.788670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c6a305-b2a0-4fc8-858a-5b1b184229fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.789317 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6a305-b2a0-4fc8-858a-5b1b184229fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.790909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c6a305-b2a0-4fc8-858a-5b1b184229fd admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.800866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1065/2133] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:05.805051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f7b3-bcd4-42df-ba6c-e37401afefac admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:05.811594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:05.818639 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f7b3-bcd4-42df-ba6c-e37401afefac admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.823308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2f93c-3f3b-46f8-a36d-9db4cb0fa9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.824023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f93c-3f3b-46f8-a36d-9db4cb0fa9db admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.824338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f93c-3f3b-46f8-a36d-9db4cb0fa9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.824613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f93c-3f3b-46f8-a36d-9db4cb0fa9db admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.825887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2f93c-3f3b-46f8-a36d-9db4cb0fa9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.827693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f2f93c-3f3b-46f8-a36d-9db4cb0fa9db admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.839541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1069/2134] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:05.843865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6a305-b2a0-4fc8-858a-5b1b184229fd admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:05.854123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c6a305-b2a0-4fc8-858a-5b1b184229fd admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:05.858612 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ee493-51b7-4995-bc71-22b8b36e7181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:05.859357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee493-51b7-4995-bc71-22b8b36e7181 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.859639 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee493-51b7-4995-bc71-22b8b36e7181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.859915 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee493-51b7-4995-bc71-22b8b36e7181 admin admin] PATH_INFO: `/v3/projects/7bc418497d6643b2b245611d0f047202/users/cc4c07e6d84e4351875efb1db8fc6ca8/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.874472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ee493-51b7-4995-bc71-22b8b36e7181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc418497d6643b2b245611d0f047202, user_id=cc4c07e6d84e4351875efb1db8fc6ca8, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:05.877241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ee493-51b7-4995-bc71-22b8b36e7181 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:05.891589 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1066/2135] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:05 2022] PUT /identity/v3/projects/7bc418497d6643b2b245611d0f047202/users/cc4c07e6d84e4351875efb1db8fc6ca8/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:05.902809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361a160-fdcb-4239-98a9-80b0206a07a5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:05.903127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361a160-fdcb-4239-98a9-80b0206a07a5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:05.903406 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361a160-fdcb-4239-98a9-80b0206a07a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:05.931551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4361a160-fdcb-4239-98a9-80b0206a07a5 None None] MFA Rules not processed for user `cc4c07e6d84e4351875efb1db8fc6ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:05.932746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4361a160-fdcb-4239-98a9-80b0206a07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:05.998510 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1070/2136] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:06.003119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ee493-51b7-4995-bc71-22b8b36e7181 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.051517 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:06.072517 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ee493-51b7-4995-bc71-22b8b36e7181 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.077532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4228a-ca3f-465a-bfbf-1670a652dff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.078253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4228a-ca3f-465a-bfbf-1670a652dff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.078610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4228a-ca3f-465a-bfbf-1670a652dff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.078963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4228a-ca3f-465a-bfbf-1670a652dff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.079810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4228a-ca3f-465a-bfbf-1670a652dff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.081788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4228a-ca3f-465a-bfbf-1670a652dff1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.109758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1067/2137] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:06 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:06.114276 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361a160-fdcb-4239-98a9-80b0206a07a5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.125346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361a160-fdcb-4239-98a9-80b0206a07a5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.184234 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6e03d-e897-4262-9a71-b9664793d6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.184853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e03d-e897-4262-9a71-b9664793d6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.185146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e03d-e897-4262-9a71-b9664793d6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.185429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e03d-e897-4262-9a71-b9664793d6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.186106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6e03d-e897-4262-9a71-b9664793d6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.188151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6e03d-e897-4262-9a71-b9664793d6d8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.214030 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1071/2138] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:06 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:06.215209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4228a-ca3f-465a-bfbf-1670a652dff1 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.229963 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4228a-ca3f-465a-bfbf-1670a652dff1 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.236091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d116f-88af-4507-a670-343b0cbae73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.236818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d116f-88af-4507-a670-343b0cbae73c admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.237191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d116f-88af-4507-a670-343b0cbae73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.237547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d116f-88af-4507-a670-343b0cbae73c admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.238847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d116f-88af-4507-a670-343b0cbae73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.241022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d116f-88af-4507-a670-343b0cbae73c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.255029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1068/2139] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:06.259175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6e03d-e897-4262-9a71-b9664793d6d8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.270693 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6e03d-e897-4262-9a71-b9664793d6d8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.286189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88fc47a-f9c6-430a-9715-efdcc246b03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.286871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fc47a-f9c6-430a-9715-efdcc246b03a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.287203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fc47a-f9c6-430a-9715-efdcc246b03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.287509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88fc47a-f9c6-430a-9715-efdcc246b03a admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.288213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fc47a-f9c6-430a-9715-efdcc246b03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.290114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88fc47a-f9c6-430a-9715-efdcc246b03a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.305436 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1072/2140] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:06.306663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d116f-88af-4507-a670-343b0cbae73c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.318889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d116f-88af-4507-a670-343b0cbae73c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.324271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946adc8-0802-41ea-a686-8fd55c5f5f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.325092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946adc8-0802-41ea-a686-8fd55c5f5f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.325478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946adc8-0802-41ea-a686-8fd55c5f5f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.325897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946adc8-0802-41ea-a686-8fd55c5f5f09 admin admin] PATH_INFO: `/v3/projects/5044bb39dd9e4bffb55c823f5ac0affe/users/c03ca7d6411e45c2ba743b0b8235ec7f/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.345839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946adc8-0802-41ea-a686-8fd55c5f5f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5044bb39dd9e4bffb55c823f5ac0affe, user_id=c03ca7d6411e45c2ba743b0b8235ec7f, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.349183 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946adc8-0802-41ea-a686-8fd55c5f5f09 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.364577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1069/2141] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:06 2022] PUT /identity/v3/projects/5044bb39dd9e4bffb55c823f5ac0affe/users/c03ca7d6411e45c2ba743b0b8235ec7f/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:06.432773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe59ba-2d49-41fc-a2e0-337366b4efee None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.433154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe59ba-2d49-41fc-a2e0-337366b4efee None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.433382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe59ba-2d49-41fc-a2e0-337366b4efee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.467994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7afe59ba-2d49-41fc-a2e0-337366b4efee None None] MFA Rules not processed for user `c03ca7d6411e45c2ba743b0b8235ec7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:06.470240 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7afe59ba-2d49-41fc-a2e0-337366b4efee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:06.526232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:06.552559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1073/2142] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:06.554386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946adc8-0802-41ea-a686-8fd55c5f5f09 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.649151 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946adc8-0802-41ea-a686-8fd55c5f5f09 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.657622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96334a2a-4b8c-4014-8543-f7dcc10ab73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.658451 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334a2a-4b8c-4014-8543-f7dcc10ab73e admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.658899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334a2a-4b8c-4014-8543-f7dcc10ab73e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.659611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96334a2a-4b8c-4014-8543-f7dcc10ab73e admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.660627 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96334a2a-4b8c-4014-8543-f7dcc10ab73e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.663596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96334a2a-4b8c-4014-8543-f7dcc10ab73e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.698570 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1070/2143] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:06 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:06.703027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe59ba-2d49-41fc-a2e0-337366b4efee None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.716040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe59ba-2d49-41fc-a2e0-337366b4efee None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.789063 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5ef4a-6570-49cc-815e-0ba38cca0bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.789663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5ef4a-6570-49cc-815e-0ba38cca0bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.789956 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5ef4a-6570-49cc-815e-0ba38cca0bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.790425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5ef4a-6570-49cc-815e-0ba38cca0bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.791106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5ef4a-6570-49cc-815e-0ba38cca0bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.793745 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5ef4a-6570-49cc-815e-0ba38cca0bf4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.826501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1074/2144] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:06 2022] POST /identity/v3/users => generated 295 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:06.827694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96334a2a-4b8c-4014-8543-f7dcc10ab73e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.838671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96334a2a-4b8c-4014-8543-f7dcc10ab73e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.843295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd5507-9637-44ef-ba00-a8eeeb5ab924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.843899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd5507-9637-44ef-ba00-a8eeeb5ab924 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.844201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd5507-9637-44ef-ba00-a8eeeb5ab924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.844458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd5507-9637-44ef-ba00-a8eeeb5ab924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.845099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd5507-9637-44ef-ba00-a8eeeb5ab924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.846773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd5507-9637-44ef-ba00-a8eeeb5ab924 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.858223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1071/2145] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:06.862311 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5ef4a-6570-49cc-815e-0ba38cca0bf4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.872552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5ef4a-6570-49cc-815e-0ba38cca0bf4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:06.877177 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0224e3a-f971-45d8-99b5-1d74874423f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:06.877840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0224e3a-f971-45d8-99b5-1d74874423f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:06.878124 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0224e3a-f971-45d8-99b5-1d74874423f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:06.878494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0224e3a-f971-45d8-99b5-1d74874423f1 admin admin] PATH_INFO: `/v3/projects/8dcddb04c8fd45188c6a8928074a0e7f/users/644a90a6456c412b8803f6ca2905daeb/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:06.893905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0224e3a-f971-45d8-99b5-1d74874423f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dcddb04c8fd45188c6a8928074a0e7f, user_id=644a90a6456c412b8803f6ca2905daeb, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:06.897055 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0224e3a-f971-45d8-99b5-1d74874423f1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:06.915628 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1075/2146] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:06 2022] PUT /identity/v3/projects/8dcddb04c8fd45188c6a8928074a0e7f/users/644a90a6456c412b8803f6ca2905daeb/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:06.919423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd5507-9637-44ef-ba00-a8eeeb5ab924 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:06.930477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd5507-9637-44ef-ba00-a8eeeb5ab924 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:07.062004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d88777-cf9e-4e7f-b4a9-b399ef7c0bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:07.062867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d88777-cf9e-4e7f-b4a9-b399ef7c0bca admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:07.063177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d88777-cf9e-4e7f-b4a9-b399ef7c0bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:07.063647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d88777-cf9e-4e7f-b4a9-b399ef7c0bca admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:07.064329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d88777-cf9e-4e7f-b4a9-b399ef7c0bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:07.069199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d88777-cf9e-4e7f-b4a9-b399ef7c0bca admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:07.090294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1072/2147] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:07.094048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0224e3a-f971-45d8-99b5-1d74874423f1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:07.109330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0224e3a-f971-45d8-99b5-1d74874423f1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:07.121629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:07.121924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:07.122261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:07.122533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] PATH_INFO: `/v3/projects/8dcddb04c8fd45188c6a8928074a0e7f/users/644a90a6456c412b8803f6ca2905daeb/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:07.124667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dcddb04c8fd45188c6a8928074a0e7f, user_id=644a90a6456c412b8803f6ca2905daeb, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:07.127610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:07.155137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1076/2148] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:07 2022] PUT /identity/v3/projects/8dcddb04c8fd45188c6a8928074a0e7f/users/644a90a6456c412b8803f6ca2905daeb/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:07.166462 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e118c-dd0e-4935-b0f3-d722efe73e3d None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:07.166785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e118c-dd0e-4935-b0f3-d722efe73e3d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:07.167024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e118c-dd0e-4935-b0f3-d722efe73e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:07.180530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:07.210961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-644e118c-dd0e-4935-b0f3-d722efe73e3d None None] MFA Rules not processed for user `644a90a6456c412b8803f6ca2905daeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:07.212282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-644e118c-dd0e-4935-b0f3-d722efe73e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:07.289199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1073/2149] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:07.300659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:07.302411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:07.363575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf1d84e-1c61-472e-b6d4-f0e8a483c87c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:07.367277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05853c8-1157-4d9c-8390-b17dbd9ea879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:07.367890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05853c8-1157-4d9c-8390-b17dbd9ea879 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:07.368217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05853c8-1157-4d9c-8390-b17dbd9ea879 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:07.368501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05853c8-1157-4d9c-8390-b17dbd9ea879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:07.426220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:07.429228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05853c8-1157-4d9c-8390-b17dbd9ea879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:07.430783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05853c8-1157-4d9c-8390-b17dbd9ea879 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:07.444446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1077/2150] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:09.004931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e118c-dd0e-4935-b0f3-d722efe73e3d None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:09.006355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-644e118c-dd0e-4935-b0f3-d722efe73e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:09.018391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e118c-dd0e-4935-b0f3-d722efe73e3d None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:09.022582 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:09.023214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:09.023506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:09.023781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:09.080575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:09.083187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:09.092341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1074/2151] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:09.605347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05853c8-1157-4d9c-8390-b17dbd9ea879 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:09.613942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05853c8-1157-4d9c-8390-b17dbd9ea879 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:09.617552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647f3aa-bf2b-4fc6-8d70-38a51e32ef74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:09.618146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647f3aa-bf2b-4fc6-8d70-38a51e32ef74 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:09.618430 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647f3aa-bf2b-4fc6-8d70-38a51e32ef74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:09.618703 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647f3aa-bf2b-4fc6-8d70-38a51e32ef74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:09.627565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647f3aa-bf2b-4fc6-8d70-38a51e32ef74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:09.629656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647f3aa-bf2b-4fc6-8d70-38a51e32ef74 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:09.639272 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1078/2152] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:12.269143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:12.270784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:12.330890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cea7632-a1fa-41a3-ac16-6b878f5006e4 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:12.335178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c607da0-a8bb-4f17-af45-d785a63a8db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:12.335783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c607da0-a8bb-4f17-af45-d785a63a8db9 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:12.336201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c607da0-a8bb-4f17-af45-d785a63a8db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:12.336320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c607da0-a8bb-4f17-af45-d785a63a8db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:12.338082 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c607da0-a8bb-4f17-af45-d785a63a8db9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:12.383868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:12.405328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c607da0-a8bb-4f17-af45-d785a63a8db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:12.406801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c607da0-a8bb-4f17-af45-d785a63a8db9 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:12.423255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1075/2153] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:13.617315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647f3aa-bf2b-4fc6-8d70-38a51e32ef74 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:13.630392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647f3aa-bf2b-4fc6-8d70-38a51e32ef74 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:13.636559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:13.637389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:13.637747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:13.638103 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] PATH_INFO: `/v3/users/cc4c07e6d84e4351875efb1db8fc6ca8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:13.639579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4c07e6d84e4351875efb1db8fc6ca8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:13.641965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:13.705069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4c07e6d84e4351875efb1db8fc6ca8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:13.721120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4c07e6d84e4351875efb1db8fc6ca8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:13.721960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4c07e6d84e4351875efb1db8fc6ca8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:13.732259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4c07e6d84e4351875efb1db8fc6ca8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:13.733046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4c07e6d84e4351875efb1db8fc6ca8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:13.749015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1079/2154] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:13 2022] DELETE /identity/v3/users/cc4c07e6d84e4351875efb1db8fc6ca8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:13.754276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8849f-fa2f-47ad-80a0-9fd5d7f7b6e4 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:13.754276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8849f-fa2f-47ad-80a0-9fd5d7f7b6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:13.755389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8849f-fa2f-47ad-80a0-9fd5d7f7b6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:13.781526 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-05c8849f-fa2f-47ad-80a0-9fd5d7f7b6e4 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:13.782775 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-05c8849f-fa2f-47ad-80a0-9fd5d7f7b6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:13.856317 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1076/2155] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:13.866537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:13.868041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:13.937004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79621151-c2a8-4ead-bbeb-7e4accdd0a38 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:13.942018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92f04d-2d15-4bb2-a9b0-58156d0f4f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:13.942756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f04d-2d15-4bb2-a9b0-58156d0f4f4d service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:13.943033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f04d-2d15-4bb2-a9b0-58156d0f4f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:13.943596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f04d-2d15-4bb2-a9b0-58156d0f4f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:13.953756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f04d-2d15-4bb2-a9b0-58156d0f4f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:13.955489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f04d-2d15-4bb2-a9b0-58156d0f4f4d service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:13.967683 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1080/2156] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:14.307707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8849f-fa2f-47ad-80a0-9fd5d7f7b6e4 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:14.309874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-05c8849f-fa2f-47ad-80a0-9fd5d7f7b6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:14.333747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8849f-fa2f-47ad-80a0-9fd5d7f7b6e4 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:14.351045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:14.351809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:14.352126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:14.352872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] PATH_INFO: `/v3/projects/7bc418497d6643b2b245611d0f047202` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:14.354258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc418497d6643b2b245611d0f047202)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:14.356360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:14.415308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc418497d6643b2b245611d0f047202', 'request_id': 'req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.428167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc418497d6643b2b245611d0f047202', 'request_id': 'req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.438588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc418497d6643b2b245611d0f047202', 'request_id': 'req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.452024 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] The token cache is being invalidate because project 7bc418497d6643b2b245611d0f047202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:14.452312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.453694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.459095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1077/2157] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:14 2022] DELETE /identity/v3/projects/7bc418497d6643b2b245611d0f047202 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:14.460438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92f04d-2d15-4bb2-a9b0-58156d0f4f4d service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:14.473849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92f04d-2d15-4bb2-a9b0-58156d0f4f4d service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:14.534634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:14.542364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:14.544421 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:14.544931 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:14.546194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] PATH_INFO: `/v3/users/c03ca7d6411e45c2ba743b0b8235ec7f` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:14.548137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03ca7d6411e45c2ba743b0b8235ec7f)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:14.551880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:14.626742 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03ca7d6411e45c2ba743b0b8235ec7f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.660767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03ca7d6411e45c2ba743b0b8235ec7f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.661819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03ca7d6411e45c2ba743b0b8235ec7f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.673725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03ca7d6411e45c2ba743b0b8235ec7f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.674606 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03ca7d6411e45c2ba743b0b8235ec7f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:14.693679 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1081/2158] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:14 2022] DELETE /identity/v3/users/c03ca7d6411e45c2ba743b0b8235ec7f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:15.184131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:15.186702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:15.202873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501f55f3-3a71-4a8e-b3bf-81ce5b2a48b1 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:15.287875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:15.288626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:15.288979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:15.289340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] PATH_INFO: `/v3/projects/5044bb39dd9e4bffb55c823f5ac0affe` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:15.290713 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5044bb39dd9e4bffb55c823f5ac0affe)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:15.293733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:15.297629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:15.355802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5044bb39dd9e4bffb55c823f5ac0affe', 'request_id': 'req-17e3749c-6c98-460c-a011-39ffeed70bbb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.362448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5044bb39dd9e4bffb55c823f5ac0affe', 'request_id': 'req-17e3749c-6c98-460c-a011-39ffeed70bbb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.372296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5044bb39dd9e4bffb55c823f5ac0affe', 'request_id': 'req-17e3749c-6c98-460c-a011-39ffeed70bbb'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.387659 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] The token cache is being invalidate because project 5044bb39dd9e4bffb55c823f5ac0affe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:15.388010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.388703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.390507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1078/2159] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:15 2022] DELETE /identity/v3/projects/5044bb39dd9e4bffb55c823f5ac0affe => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:15.395369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:15.397083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:15.411022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8276368c-b3d9-4654-b936-b03b339cf449 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:15.492410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:15.492765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:15.492983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:15.493663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] PATH_INFO: `/v3/users/644a90a6456c412b8803f6ca2905daeb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:15.494472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=644a90a6456c412b8803f6ca2905daeb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:15.496433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:15.556858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '644a90a6456c412b8803f6ca2905daeb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.572892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '644a90a6456c412b8803f6ca2905daeb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.573774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '644a90a6456c412b8803f6ca2905daeb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.581564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '644a90a6456c412b8803f6ca2905daeb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.582882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '644a90a6456c412b8803f6ca2905daeb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:15.588489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:15.597446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1082/2160] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:15 2022] DELETE /identity/v3/users/644a90a6456c412b8803f6ca2905daeb => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:15.961768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:15.963425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:15.981704 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:15.982798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3749c-6c98-460c-a011-39ffeed70bbb admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:15.983202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.100405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.101917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.102437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.102898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] PATH_INFO: `/v3/projects/8dcddb04c8fd45188c6a8928074a0e7f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.104256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dcddb04c8fd45188c6a8928074a0e7f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.106448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.106583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0946053-3876-4bff-bcc6-7cca138c3b4f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.119672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.120581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.121571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.121860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.123925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.140944 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.142606 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.155708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1083/2161] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:19:16.156737 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.158948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.180278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fd184-0ae0-40a4-ae64-e1817b40bbdc service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.194194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dcddb04c8fd45188c6a8928074a0e7f', 'request_id': 'req-3858a2c7-98c7-484e-a4e9-02d80badb019'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.201564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dcddb04c8fd45188c6a8928074a0e7f', 'request_id': 'req-3858a2c7-98c7-484e-a4e9-02d80badb019'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.211808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dcddb04c8fd45188c6a8928074a0e7f', 'request_id': 'req-3858a2c7-98c7-484e-a4e9-02d80badb019'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.226850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] The token cache is being invalidate because project 8dcddb04c8fd45188c6a8928074a0e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:16.227263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.228142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-3858a2c7-98c7-484e-a4e9-02d80badb019 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.230194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1079/2162] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:15 2022] DELETE /identity/v3/projects/8dcddb04c8fd45188c6a8928074a0e7f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:16.234872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cffb3-58ee-4dd3-ad45-dd020a1d4d84 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.235187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cffb3-58ee-4dd3-ad45-dd020a1d4d84 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.235657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cffb3-58ee-4dd3-ad45-dd020a1d4d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.248782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.249420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.249702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.249980 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] PATH_INFO: `/v3/users/c5619c8183e04e1d9073f1b40e057967` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.251135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5619c8183e04e1d9073f1b40e057967)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.253065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.259232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-531cffb3-58ee-4dd3-ad45-dd020a1d4d84 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:16.259232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-531cffb3-58ee-4dd3-ad45-dd020a1d4d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.268610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1080/2163] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:16.280597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a504-e7b6-4ffb-b669-ad5d1545c88e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.280882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a504-e7b6-4ffb-b669-ad5d1545c88e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.281217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee1a504-e7b6-4ffb-b669-ad5d1545c88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.301672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-cee1a504-e7b6-4ffb-b669-ad5d1545c88e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:16.302909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-cee1a504-e7b6-4ffb-b669-ad5d1545c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.307751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5619c8183e04e1d9073f1b40e057967'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.311471 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1081/2164] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:16.315546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a504-e7b6-4ffb-b669-ad5d1545c88e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.321218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5619c8183e04e1d9073f1b40e057967'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.321862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5619c8183e04e1d9073f1b40e057967'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.326375 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee1a504-e7b6-4ffb-b669-ad5d1545c88e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.329763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5619c8183e04e1d9073f1b40e057967'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.330466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5619c8183e04e1d9073f1b40e057967'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.332396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.333300 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.333586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.334035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.334682 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:16.334993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.336611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.339346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:16.343744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1084/2165] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:16 2022] DELETE /identity/v3/users/c5619c8183e04e1d9073f1b40e057967 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:16.352748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1082/2166] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:19:16.356949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.358564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.365581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:16.428719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a02f84-a069-4f8c-9449-8e8c7221b468 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.433514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.434132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.434419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.434699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.435385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.437305 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.467766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1085/2167] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:16.468993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.479989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202e7028-9e09-4376-8b86-dd0f236bc5c7 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.524617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.527101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.540269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ed6d3-5dea-46ab-b821-19a9f283d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.540979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ed6d3-5dea-46ab-b821-19a9f283d843 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.541333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ed6d3-5dea-46ab-b821-19a9f283d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.541680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ed6d3-5dea-46ab-b821-19a9f283d843 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.542443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ed6d3-5dea-46ab-b821-19a9f283d843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.545184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ed6d3-5dea-46ab-b821-19a9f283d843 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.572187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1083/2168] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:16.573241 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ed6d3-5dea-46ab-b821-19a9f283d843 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.583557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ed6d3-5dea-46ab-b821-19a9f283d843 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.588377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d305c-0c69-4e1d-a95b-365596492212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.589053 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d305c-0c69-4e1d-a95b-365596492212 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.589401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d305c-0c69-4e1d-a95b-365596492212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.589682 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d305c-0c69-4e1d-a95b-365596492212 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.590330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d305c-0c69-4e1d-a95b-365596492212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.591977 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d305c-0c69-4e1d-a95b-365596492212 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.603347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1084/2169] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:16.603514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae3c5b-81dc-4488-ac5c-2eb5705aa2f5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.607204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d305c-0c69-4e1d-a95b-365596492212 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.611135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.611776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.612061 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.612342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] PATH_INFO: `/v3/projects/69beb82dd21b447d8a784e0570c4323c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.613919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=69beb82dd21b447d8a784e0570c4323c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.615925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.617126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d305c-0c69-4e1d-a95b-365596492212 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.622138 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98671eee-7e70-4016-a6f0-8227a1918795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.622797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98671eee-7e70-4016-a6f0-8227a1918795 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.623115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98671eee-7e70-4016-a6f0-8227a1918795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.623397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98671eee-7e70-4016-a6f0-8227a1918795 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.624035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98671eee-7e70-4016-a6f0-8227a1918795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.625609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98671eee-7e70-4016-a6f0-8227a1918795 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.636534 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1085/2170] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:16.640340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98671eee-7e70-4016-a6f0-8227a1918795 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.651492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98671eee-7e70-4016-a6f0-8227a1918795 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.666523 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69beb82dd21b447d8a784e0570c4323c', 'request_id': 'req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.672832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69beb82dd21b447d8a784e0570c4323c', 'request_id': 'req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.680969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69beb82dd21b447d8a784e0570c4323c', 'request_id': 'req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.692286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] The token cache is being invalidate because project 69beb82dd21b447d8a784e0570c4323c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:16.692601 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.693253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-70353bbd-9726-4e17-b8e8-3e9fdd9c5d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:16.694829 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1086/2171] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:16 2022] DELETE /identity/v3/projects/69beb82dd21b447d8a784e0570c4323c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:16.699461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b38e29-3223-4d48-9e73-1c604e6160c6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.699748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b38e29-3223-4d48-9e73-1c604e6160c6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.700023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b38e29-3223-4d48-9e73-1c604e6160c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.710746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a334b29-55ae-426b-a13f-c63edfd68c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.711443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a334b29-55ae-426b-a13f-c63edfd68c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.711729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a334b29-55ae-426b-a13f-c63edfd68c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.712011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a334b29-55ae-426b-a13f-c63edfd68c9a admin admin] PATH_INFO: `/v3/projects/f0bee9967bdb41eca0acff51d5e105e9/users/af98e8f344914ae59852d82d176b4dc2/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.720868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a8b38e29-3223-4d48-9e73-1c604e6160c6 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:16.722733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a8b38e29-3223-4d48-9e73-1c604e6160c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.728665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a334b29-55ae-426b-a13f-c63edfd68c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0bee9967bdb41eca0acff51d5e105e9, user_id=af98e8f344914ae59852d82d176b4dc2, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.730837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1087/2172] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:16.731518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a334b29-55ae-426b-a13f-c63edfd68c9a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.739281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef2939-9bbf-428a-8e3b-b14a0b2e9ced None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.739573 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef2939-9bbf-428a-8e3b-b14a0b2e9ced None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.739995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef2939-9bbf-428a-8e3b-b14a0b2e9ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.744999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1086/2173] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:16 2022] PUT /identity/v3/projects/f0bee9967bdb41eca0acff51d5e105e9/users/af98e8f344914ae59852d82d176b4dc2/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:16.749969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27cb6f-3422-4180-a4ea-58eef34ecfd8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.750258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27cb6f-3422-4180-a4ea-58eef34ecfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.750538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27cb6f-3422-4180-a4ea-58eef34ecfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.762135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5fef2939-9bbf-428a-8e3b-b14a0b2e9ced None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:16.763377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5fef2939-9bbf-428a-8e3b-b14a0b2e9ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.769767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7c27cb6f-3422-4180-a4ea-58eef34ecfd8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:16.771031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c27cb6f-3422-4180-a4ea-58eef34ecfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.772700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:16.815542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:16.829891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1088/2174] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:16.834588 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef2939-9bbf-428a-8e3b-b14a0b2e9ced None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.836209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1087/2175] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:16.843760 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef2939-9bbf-428a-8e3b-b14a0b2e9ced None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.846485 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27cb6f-3422-4180-a4ea-58eef34ecfd8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.848051 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7c27cb6f-3422-4180-a4ea-58eef34ecfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:16.848700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.849304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.849591 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.849870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.850521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:16.850832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.852465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.867940 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1089/2176] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:19:16.868878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.878964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb37a0c-aec7-4d15-bf85-f238dec317d9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.883873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68ac48-c238-488f-a0d2-5f5cc140c5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.884458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68ac48-c238-488f-a0d2-5f5cc140c5ad admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.884743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68ac48-c238-488f-a0d2-5f5cc140c5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.885023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a68ac48-c238-488f-a0d2-5f5cc140c5ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.886040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68ac48-c238-488f-a0d2-5f5cc140c5ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.887987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a68ac48-c238-488f-a0d2-5f5cc140c5ad admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.907833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27cb6f-3422-4180-a4ea-58eef34ecfd8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.911952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb9023-3087-4798-93ce-0f8a7bad6aca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.912566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb9023-3087-4798-93ce-0f8a7bad6aca service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.912849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb9023-3087-4798-93ce-0f8a7bad6aca service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.913130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb9023-3087-4798-93ce-0f8a7bad6aca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.919838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1090/2177] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:16.921105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68ac48-c238-488f-a0d2-5f5cc140c5ad admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:16.926319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb9023-3087-4798-93ce-0f8a7bad6aca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.927797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bb9023-3087-4798-93ce-0f8a7bad6aca service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:16.937686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1088/2178] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:19:16.985660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a68ac48-c238-488f-a0d2-5f5cc140c5ad admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:16.993015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9a5e-a55c-429b-92ab-6814bb530189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:16.993781 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9a5e-a55c-429b-92ab-6814bb530189 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:16.994076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9a5e-a55c-429b-92ab-6814bb530189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:16.994352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c9a5e-a55c-429b-92ab-6814bb530189 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:16.995173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c9a5e-a55c-429b-92ab-6814bb530189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:16.997145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c9a5e-a55c-429b-92ab-6814bb530189 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.019874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1091/2179] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:16 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:17.024469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb9023-3087-4798-93ce-0f8a7bad6aca service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.035085 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb9023-3087-4798-93ce-0f8a7bad6aca service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.039922 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5f5ec-b610-4a04-b695-b909c4a90d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.040638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5f5ec-b610-4a04-b695-b909c4a90d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.040964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5f5ec-b610-4a04-b695-b909c4a90d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.041283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5f5ec-b610-4a04-b695-b909c4a90d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.041928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5f5ec-b610-4a04-b695-b909c4a90d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.043559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5f5ec-b610-4a04-b695-b909c4a90d21 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.054374 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1089/2180] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:17.058627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9a5e-a55c-429b-92ab-6814bb530189 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.073261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c9a5e-a55c-429b-92ab-6814bb530189 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.079443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924582a-e9ac-44c5-b806-fb2708c88c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.080142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924582a-e9ac-44c5-b806-fb2708c88c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.081117 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924582a-e9ac-44c5-b806-fb2708c88c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.081472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924582a-e9ac-44c5-b806-fb2708c88c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.082656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924582a-e9ac-44c5-b806-fb2708c88c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.084569 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924582a-e9ac-44c5-b806-fb2708c88c02 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.098720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1092/2181] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:17.100086 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5f5ec-b610-4a04-b695-b909c4a90d21 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.110434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5f5ec-b610-4a04-b695-b909c4a90d21 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.116562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701d00-5c10-4bcf-bf74-af36a5c7be66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.117427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701d00-5c10-4bcf-bf74-af36a5c7be66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.117732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701d00-5c10-4bcf-bf74-af36a5c7be66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.118178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd701d00-5c10-4bcf-bf74-af36a5c7be66 admin admin] PATH_INFO: `/v3/projects/eade7ab39f704a0db41f5af24d58e3cf/users/116fb631bda24f9ab69876eccbec582f/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.137345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701d00-5c10-4bcf-bf74-af36a5c7be66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eade7ab39f704a0db41f5af24d58e3cf, user_id=116fb631bda24f9ab69876eccbec582f, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.140089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd701d00-5c10-4bcf-bf74-af36a5c7be66 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.154661 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1090/2182] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:17 2022] PUT /identity/v3/projects/eade7ab39f704a0db41f5af24d58e3cf/users/116fb631bda24f9ab69876eccbec582f/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:17.165274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24d9b-d21e-4a51-9de9-d446d12d3e53 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.165900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24d9b-d21e-4a51-9de9-d446d12d3e53 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.166454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a24d9b-d21e-4a51-9de9-d446d12d3e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.201309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a6a24d9b-d21e-4a51-9de9-d446d12d3e53 None None] MFA Rules not processed for user `116fb631bda24f9ab69876eccbec582f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:17.202592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a6a24d9b-d21e-4a51-9de9-d446d12d3e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:17.271335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:17.299904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1093/2183] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:17.305101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701d00-5c10-4bcf-bf74-af36a5c7be66 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.384883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd701d00-5c10-4bcf-bf74-af36a5c7be66 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.393229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d91f6-94b0-4811-a462-e99de7a61d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.393897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d91f6-94b0-4811-a462-e99de7a61d1a admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.394182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d91f6-94b0-4811-a462-e99de7a61d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.394461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d91f6-94b0-4811-a462-e99de7a61d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.395146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d91f6-94b0-4811-a462-e99de7a61d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.397071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d91f6-94b0-4811-a462-e99de7a61d1a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.426262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1091/2184] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:17 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:17.430688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24d9b-d21e-4a51-9de9-d446d12d3e53 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.442026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a24d9b-d21e-4a51-9de9-d446d12d3e53 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.500848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce4738-4163-4373-8e18-52ccc7c9352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.501463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce4738-4163-4373-8e18-52ccc7c9352a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.501748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce4738-4163-4373-8e18-52ccc7c9352a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.502025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce4738-4163-4373-8e18-52ccc7c9352a admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.502673 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce4738-4163-4373-8e18-52ccc7c9352a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.504689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce4738-4163-4373-8e18-52ccc7c9352a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.532226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1094/2185] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:17 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:17.533537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d91f6-94b0-4811-a462-e99de7a61d1a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.545013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d91f6-94b0-4811-a462-e99de7a61d1a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.551004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d9ac0-5fba-48d5-beb4-1f0e0795d863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.551810 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d9ac0-5fba-48d5-beb4-1f0e0795d863 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.552195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d9ac0-5fba-48d5-beb4-1f0e0795d863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.552556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d9ac0-5fba-48d5-beb4-1f0e0795d863 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.553324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d9ac0-5fba-48d5-beb4-1f0e0795d863 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.555133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d9ac0-5fba-48d5-beb4-1f0e0795d863 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.571980 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1092/2186] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:17.573325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce4738-4163-4373-8e18-52ccc7c9352a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.586996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce4738-4163-4373-8e18-52ccc7c9352a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.596349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5488-06e7-4745-8357-d60ec7989931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.597676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5488-06e7-4745-8357-d60ec7989931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.598029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5488-06e7-4745-8357-d60ec7989931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.598413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a5488-06e7-4745-8357-d60ec7989931 admin admin] PATH_INFO: `/v3/projects/04e8f63e4d6f402b9ca0ae73769c7839/users/ef12457ac3e34a36bb169646b8fdd75e/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.616143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a5488-06e7-4745-8357-d60ec7989931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e8f63e4d6f402b9ca0ae73769c7839, user_id=ef12457ac3e34a36bb169646b8fdd75e, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.619331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a5488-06e7-4745-8357-d60ec7989931 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.640724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1095/2187] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:17 2022] PUT /identity/v3/projects/04e8f63e4d6f402b9ca0ae73769c7839/users/ef12457ac3e34a36bb169646b8fdd75e/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:17.642055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d9ac0-5fba-48d5-beb4-1f0e0795d863 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.654157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d9ac0-5fba-48d5-beb4-1f0e0795d863 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.686541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:17.725385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d9c65-7356-4fb9-a574-5089a27619ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.726311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d9c65-7356-4fb9-a574-5089a27619ef admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.726978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d9c65-7356-4fb9-a574-5089a27619ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.728201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d9c65-7356-4fb9-a574-5089a27619ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.729124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d9c65-7356-4fb9-a574-5089a27619ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.732008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d9c65-7356-4fb9-a574-5089a27619ef admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.748505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1093/2188] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:17.749863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5488-06e7-4745-8357-d60ec7989931 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.762763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a5488-06e7-4745-8357-d60ec7989931 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:17.768610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:17.770381 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.770775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.771159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] PATH_INFO: `/v3/projects/04e8f63e4d6f402b9ca0ae73769c7839/users/ef12457ac3e34a36bb169646b8fdd75e/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.773292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e8f63e4d6f402b9ca0ae73769c7839, user_id=ef12457ac3e34a36bb169646b8fdd75e, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:17.776119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:17.789281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1096/2189] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:17 2022] PUT /identity/v3/projects/04e8f63e4d6f402b9ca0ae73769c7839/users/ef12457ac3e34a36bb169646b8fdd75e/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:17.802227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9ad7-4a40-412c-8c0d-d39cea4804ac None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:17.802577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9ad7-4a40-412c-8c0d-d39cea4804ac None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:17.802781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9ad7-4a40-412c-8c0d-d39cea4804ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:17.836683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5b0d9ad7-4a40-412c-8c0d-d39cea4804ac None None] MFA Rules not processed for user `ef12457ac3e34a36bb169646b8fdd75e`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:17.838114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5b0d9ad7-4a40-412c-8c0d-d39cea4804ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:17.924152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1094/2190] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:17.937734 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:17.940278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:18.009807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098ba16-5d6c-479e-b119-7762383a0c79 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:18.014343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0152e-3b49-439b-9f33-636cc33f4a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:18.015082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0152e-3b49-439b-9f33-636cc33f4a95 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:18.015444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0152e-3b49-439b-9f33-636cc33f4a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:18.015791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0152e-3b49-439b-9f33-636cc33f4a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:18.026762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0152e-3b49-439b-9f33-636cc33f4a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:18.028384 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0152e-3b49-439b-9f33-636cc33f4a95 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:18.045753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1097/2191] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:18.598263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d9ad7-4a40-412c-8c0d-d39cea4804ac None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:18.724259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d9ad7-4a40-412c-8c0d-d39cea4804ac None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:18.730944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:18.731687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:18.734911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:18.734911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] PATH_INFO: `/v3/users/116fb631bda24f9ab69876eccbec582f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:18.737640 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=116fb631bda24f9ab69876eccbec582f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:18.739880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:18.810835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116fb631bda24f9ab69876eccbec582f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:18.819015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:18.837837 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116fb631bda24f9ab69876eccbec582f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:18.839588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116fb631bda24f9ab69876eccbec582f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:18.852385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116fb631bda24f9ab69876eccbec582f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:18.861937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116fb631bda24f9ab69876eccbec582f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:18.871665 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1095/2192] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:18 2022] DELETE /identity/v3/users/116fb631bda24f9ab69876eccbec582f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:18.879572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa885c7-4637-48b7-9059-8e05d28f9825 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:18.879984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa885c7-4637-48b7-9059-8e05d28f9825 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:18.880318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa885c7-4637-48b7-9059-8e05d28f9825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:18.902353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7fa885c7-4637-48b7-9059-8e05d28f9825 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:18.903646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7fa885c7-4637-48b7-9059-8e05d28f9825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:18.949632 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:18.974065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1098/2193] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:18.984178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:18.986045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:19.067363 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e13a9-dd61-4b0e-b34f-034c5dc4dce7 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:19.071308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f74b6-add4-4fec-a82c-84fd65a65ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:19.072190 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f74b6-add4-4fec-a82c-84fd65a65ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:19.072568 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f74b6-add4-4fec-a82c-84fd65a65ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:19.072937 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f74b6-add4-4fec-a82c-84fd65a65ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:19.085664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f74b6-add4-4fec-a82c-84fd65a65ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:19.087240 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f74b6-add4-4fec-a82c-84fd65a65ea3 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:19.097786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1096/2194] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:19.471235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa885c7-4637-48b7-9059-8e05d28f9825 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:19.476168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7fa885c7-4637-48b7-9059-8e05d28f9825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:19.509785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa885c7-4637-48b7-9059-8e05d28f9825 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:19.517569 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:19.518354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:19.518487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:19.519325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] PATH_INFO: `/v3/projects/eade7ab39f704a0db41f5af24d58e3cf` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:19.521125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eade7ab39f704a0db41f5af24d58e3cf)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:19.523243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:19.611332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eade7ab39f704a0db41f5af24d58e3cf', 'request_id': 'req-cec36d34-22b8-493f-892f-6da4dfee5cd0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.622996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eade7ab39f704a0db41f5af24d58e3cf', 'request_id': 'req-cec36d34-22b8-493f-892f-6da4dfee5cd0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.638024 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eade7ab39f704a0db41f5af24d58e3cf', 'request_id': 'req-cec36d34-22b8-493f-892f-6da4dfee5cd0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.658915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] The token cache is being invalidate because project eade7ab39f704a0db41f5af24d58e3cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:19.659525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.661237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.667165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1099/2195] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:19 2022] DELETE /identity/v3/projects/eade7ab39f704a0db41f5af24d58e3cf => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:19.669613 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f74b6-add4-4fec-a82c-84fd65a65ea3 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:19.695463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f74b6-add4-4fec-a82c-84fd65a65ea3 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:19.787027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:19.788014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:19.788391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:19.788766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] PATH_INFO: `/v3/users/ef12457ac3e34a36bb169646b8fdd75e` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:19.790034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef12457ac3e34a36bb169646b8fdd75e)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:19.792358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:19.855928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef12457ac3e34a36bb169646b8fdd75e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.873999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef12457ac3e34a36bb169646b8fdd75e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.875923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef12457ac3e34a36bb169646b8fdd75e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.886351 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef12457ac3e34a36bb169646b8fdd75e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.890762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:19.899494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4539548-baef-44f0-9c83-6ae968406d65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef12457ac3e34a36bb169646b8fdd75e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:19.910798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1097/2196] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:19 2022] DELETE /identity/v3/users/ef12457ac3e34a36bb169646b8fdd75e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:20.435827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:20.437396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:20.472682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec36d34-22b8-493f-892f-6da4dfee5cd0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:20.572585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:20.573317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:20.574140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:20.574427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] PATH_INFO: `/v3/projects/04e8f63e4d6f402b9ca0ae73769c7839` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:20.575665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e8f63e4d6f402b9ca0ae73769c7839)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:20.578005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:20.634766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e8f63e4d6f402b9ca0ae73769c7839', 'request_id': 'req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:20.641700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e8f63e4d6f402b9ca0ae73769c7839', 'request_id': 'req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:20.650554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e8f63e4d6f402b9ca0ae73769c7839', 'request_id': 'req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:20.658213 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:20.670852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] The token cache is being invalidate because project 04e8f63e4d6f402b9ca0ae73769c7839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:20.671393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:20.672895 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8b83ce83-9f31-45ee-a0e9-1938dfc56c08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:20.679102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1100/2197] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:20 2022] DELETE /identity/v3/projects/04e8f63e4d6f402b9ca0ae73769c7839 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:20.680861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2495-e28d-4c1d-99f6-6781c3f8f1f2 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:20.681360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2495-e28d-4c1d-99f6-6781c3f8f1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:20.681458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a2495-e28d-4c1d-99f6-6781c3f8f1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:20.705229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-715a2495-e28d-4c1d-99f6-6781c3f8f1f2 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:20.707357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-715a2495-e28d-4c1d-99f6-6781c3f8f1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:20.788586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1098/2198] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:20.795909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac939d31-cbea-4b51-b561-e001db7c99e9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:20.796192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac939d31-cbea-4b51-b561-e001db7c99e9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:20.796482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac939d31-cbea-4b51-b561-e001db7c99e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:20.822655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ac939d31-cbea-4b51-b561-e001db7c99e9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:20.823519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ac939d31-cbea-4b51-b561-e001db7c99e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:20.832584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1101/2199] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:20.836763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a2495-e28d-4c1d-99f6-6781c3f8f1f2 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:20.851715 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a2495-e28d-4c1d-99f6-6781c3f8f1f2 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:20.860575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:20.861355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:20.861743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:20.862152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:20.863569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:20.866204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:20.869896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:20.896448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1099/2200] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:19:20.901256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac939d31-cbea-4b51-b561-e001db7c99e9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:20.912246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac939d31-cbea-4b51-b561-e001db7c99e9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:20.923986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa486f3-cea0-4509-a8e9-2e30c4cc018d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:20.924995 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa486f3-cea0-4509-a8e9-2e30c4cc018d admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:20.925652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa486f3-cea0-4509-a8e9-2e30c4cc018d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:20.926024 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa486f3-cea0-4509-a8e9-2e30c4cc018d admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:20.926825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa486f3-cea0-4509-a8e9-2e30c4cc018d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:20.928819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa486f3-cea0-4509-a8e9-2e30c4cc018d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:20.971416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1102/2201] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:20 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:20.976574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:20.987030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce13be-3eb8-4705-8e9c-6d71263c0eea admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.059166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d90fd0-777a-4193-b54a-3934d2003eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.059895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90fd0-777a-4193-b54a-3934d2003eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.060186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90fd0-777a-4193-b54a-3934d2003eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.060468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d90fd0-777a-4193-b54a-3934d2003eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.061112 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d90fd0-777a-4193-b54a-3934d2003eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.065376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d90fd0-777a-4193-b54a-3934d2003eaf admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.103116 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa486f3-cea0-4509-a8e9-2e30c4cc018d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.103468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1100/2202] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:20 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:21.118337 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa486f3-cea0-4509-a8e9-2e30c4cc018d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.124970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b655401-0438-4ce1-bf87-4263263f9898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.125613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b655401-0438-4ce1-bf87-4263263f9898 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.125936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b655401-0438-4ce1-bf87-4263263f9898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.126267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b655401-0438-4ce1-bf87-4263263f9898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.126966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b655401-0438-4ce1-bf87-4263263f9898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.128776 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b655401-0438-4ce1-bf87-4263263f9898 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.140521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1103/2203] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:21.145578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d90fd0-777a-4193-b54a-3934d2003eaf admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.156816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d90fd0-777a-4193-b54a-3934d2003eaf admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.162410 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac2302-cd49-475c-b97b-a739fefb195e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.163049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac2302-cd49-475c-b97b-a739fefb195e admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.163369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac2302-cd49-475c-b97b-a739fefb195e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.163676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac2302-cd49-475c-b97b-a739fefb195e admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.164357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac2302-cd49-475c-b97b-a739fefb195e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.166287 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac2302-cd49-475c-b97b-a739fefb195e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.178394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1101/2204] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:21.182612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b655401-0438-4ce1-bf87-4263263f9898 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.194066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b655401-0438-4ce1-bf87-4263263f9898 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.198852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbdc7e-dc26-450d-a0f6-79fc6f4fdd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.199584 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbdc7e-dc26-450d-a0f6-79fc6f4fdd78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.199872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbdc7e-dc26-450d-a0f6-79fc6f4fdd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.200154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dbdc7e-dc26-450d-a0f6-79fc6f4fdd78 admin admin] PATH_INFO: `/v3/projects/ddbce4474b2b4e9c9e672f76b2bcdc91/users/06c41c2b44c1434a8ee6b26580fe3f42/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.217199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbdc7e-dc26-450d-a0f6-79fc6f4fdd78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddbce4474b2b4e9c9e672f76b2bcdc91, user_id=06c41c2b44c1434a8ee6b26580fe3f42, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.220186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dbdc7e-dc26-450d-a0f6-79fc6f4fdd78 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.236544 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1104/2205] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:21 2022] PUT /identity/v3/projects/ddbce4474b2b4e9c9e672f76b2bcdc91/users/06c41c2b44c1434a8ee6b26580fe3f42/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:21.248930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8951d-dce0-4018-845f-3be098317664 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.249247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8951d-dce0-4018-845f-3be098317664 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.249474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8951d-dce0-4018-845f-3be098317664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.292606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a5d8951d-dce0-4018-845f-3be098317664 None None] MFA Rules not processed for user `06c41c2b44c1434a8ee6b26580fe3f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:21.295816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a5d8951d-dce0-4018-845f-3be098317664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:21.353819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:21.384919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1102/2206] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:21.389977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbdc7e-dc26-450d-a0f6-79fc6f4fdd78 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.517332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dbdc7e-dc26-450d-a0f6-79fc6f4fdd78 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.531568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68987d-1044-4159-9089-6c70ff6f6bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.532315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68987d-1044-4159-9089-6c70ff6f6bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.532604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68987d-1044-4159-9089-6c70ff6f6bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.532887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa68987d-1044-4159-9089-6c70ff6f6bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.533634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68987d-1044-4159-9089-6c70ff6f6bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.535750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa68987d-1044-4159-9089-6c70ff6f6bf4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.622571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1105/2207] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:21 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:21.626712 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8951d-dce0-4018-845f-3be098317664 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.654247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8951d-dce0-4018-845f-3be098317664 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.731896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff82f5-a380-4786-b2dd-29453ec30ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.732566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff82f5-a380-4786-b2dd-29453ec30ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.733694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff82f5-a380-4786-b2dd-29453ec30ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.734193 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff82f5-a380-4786-b2dd-29453ec30ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.734971 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff82f5-a380-4786-b2dd-29453ec30ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.737017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff82f5-a380-4786-b2dd-29453ec30ee3 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.762849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1103/2208] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:21 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:21.767043 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68987d-1044-4159-9089-6c70ff6f6bf4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.768997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219c3d-4b41-477a-b663-7a6c1dab3d7d None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.769300 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219c3d-4b41-477a-b663-7a6c1dab3d7d None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.769616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e219c3d-4b41-477a-b663-7a6c1dab3d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.778887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa68987d-1044-4159-9089-6c70ff6f6bf4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.784771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840553ab-7d27-4174-b935-241c3816cfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.785729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840553ab-7d27-4174-b935-241c3816cfed admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.786085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840553ab-7d27-4174-b935-241c3816cfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.786442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840553ab-7d27-4174-b935-241c3816cfed admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.787216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840553ab-7d27-4174-b935-241c3816cfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.788893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840553ab-7d27-4174-b935-241c3816cfed admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.802370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1106/2209] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:21.805023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-1e219c3d-4b41-477a-b663-7a6c1dab3d7d None None] MFA Rules not processed for user `af98e8f344914ae59852d82d176b4dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:21.806596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840553ab-7d27-4174-b935-241c3816cfed admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.806940 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1e219c3d-4b41-477a-b663-7a6c1dab3d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:21.818643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840553ab-7d27-4174-b935-241c3816cfed admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.824685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbedc9-693d-4d58-8ee0-2c5b7eef1c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.825477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbedc9-693d-4d58-8ee0-2c5b7eef1c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.825852 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbedc9-693d-4d58-8ee0-2c5b7eef1c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.826208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbedc9-693d-4d58-8ee0-2c5b7eef1c4f admin admin] PATH_INFO: `/v3/projects/d838d391f03c43138bad70d61f090a78/users/9b43413d9f3944dd890a3808ffcdd2c6/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.845878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbedc9-693d-4d58-8ee0-2c5b7eef1c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d838d391f03c43138bad70d61f090a78, user_id=9b43413d9f3944dd890a3808ffcdd2c6, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.848869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbedc9-693d-4d58-8ee0-2c5b7eef1c4f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.869757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1107/2210] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:21 2022] PUT /identity/v3/projects/d838d391f03c43138bad70d61f090a78/users/9b43413d9f3944dd890a3808ffcdd2c6/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:21.871080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbedc9-693d-4d58-8ee0-2c5b7eef1c4f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.886689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1104/2211] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:21.890679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e219c3d-4b41-477a-b663-7a6c1dab3d7d None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.892036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1e219c3d-4b41-477a-b663-7a6c1dab3d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:21.908523 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e219c3d-4b41-477a-b663-7a6c1dab3d7d None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.924200 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:21.948245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbedc9-693d-4d58-8ee0-2c5b7eef1c4f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.953215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf802d-923e-4a61-949d-10ed9dd416ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.953837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf802d-923e-4a61-949d-10ed9dd416ea admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.954456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf802d-923e-4a61-949d-10ed9dd416ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.954456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf802d-923e-4a61-949d-10ed9dd416ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.955048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf802d-923e-4a61-949d-10ed9dd416ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.956840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf802d-923e-4a61-949d-10ed9dd416ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.970713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1108/2212] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:21.971929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf802d-923e-4a61-949d-10ed9dd416ea admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:21.978779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff775c7f-5f10-499d-80b0-a8eea4d6f12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.979507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff775c7f-5f10-499d-80b0-a8eea4d6f12d admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.979859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff775c7f-5f10-499d-80b0-a8eea4d6f12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.980210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff775c7f-5f10-499d-80b0-a8eea4d6f12d admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.980893 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff775c7f-5f10-499d-80b0-a8eea4d6f12d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.982859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff775c7f-5f10-499d-80b0-a8eea4d6f12d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:21.983010 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf802d-923e-4a61-949d-10ed9dd416ea admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:21.988374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c72f-8200-4763-8674-b5ca9ccab71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:21.989346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c72f-8200-4763-8674-b5ca9ccab71b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:21.989817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c72f-8200-4763-8674-b5ca9ccab71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:21.990292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23c72f-8200-4763-8674-b5ca9ccab71b admin admin] PATH_INFO: `/v3/projects/d838d391f03c43138bad70d61f090a78/users/9b43413d9f3944dd890a3808ffcdd2c6/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:21.993565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23c72f-8200-4763-8674-b5ca9ccab71b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d838d391f03c43138bad70d61f090a78, user_id=9b43413d9f3944dd890a3808ffcdd2c6, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:21.995910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:21.998744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23c72f-8200-4763-8674-b5ca9ccab71b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.013447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1109/2213] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:21 2022] PUT /identity/v3/projects/d838d391f03c43138bad70d61f090a78/users/9b43413d9f3944dd890a3808ffcdd2c6/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:22.020381 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1105/2214] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:21 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:22.023817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c72f-8200-4763-8674-b5ca9ccab71b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:22.024274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0f3fb-75aa-4a8b-ad52-12d14a0ec594 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.024567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0f3fb-75aa-4a8b-ad52-12d14a0ec594 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.024850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0f3fb-75aa-4a8b-ad52-12d14a0ec594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.068395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-3df0f3fb-75aa-4a8b-ad52-12d14a0ec594 None None] MFA Rules not processed for user `9b43413d9f3944dd890a3808ffcdd2c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:22.069685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3df0f3fb-75aa-4a8b-ad52-12d14a0ec594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:22.119342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23c72f-8200-4763-8674-b5ca9ccab71b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:22.164809 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1106/2215] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:22.177417 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0f3fb-75aa-4a8b-ad52-12d14a0ec594 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:22.178793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3df0f3fb-75aa-4a8b-ad52-12d14a0ec594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:22.195470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4072768-d65b-42fe-a774-9e51ebe996f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:22.196330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072768-d65b-42fe-a774-9e51ebe996f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.196618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072768-d65b-42fe-a774-9e51ebe996f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.197029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4072768-d65b-42fe-a774-9e51ebe996f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.197676 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4072768-d65b-42fe-a774-9e51ebe996f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:22.199677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4072768-d65b-42fe-a774-9e51ebe996f5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.241459 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1110/2216] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:22 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:22.245654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4072768-d65b-42fe-a774-9e51ebe996f5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:22.256990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4072768-d65b-42fe-a774-9e51ebe996f5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:22.261999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938914cf-96f2-4c16-9cc0-30cee0fbd9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:22.262600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938914cf-96f2-4c16-9cc0-30cee0fbd9bc admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.262884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938914cf-96f2-4c16-9cc0-30cee0fbd9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.263222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0f3fb-75aa-4a8b-ad52-12d14a0ec594 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:22.263360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938914cf-96f2-4c16-9cc0-30cee0fbd9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.263877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938914cf-96f2-4c16-9cc0-30cee0fbd9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:22.265461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938914cf-96f2-4c16-9cc0-30cee0fbd9bc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.267716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfcc77-3395-40ba-9b67-2afde54e5888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:22.268442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfcc77-3395-40ba-9b67-2afde54e5888 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.268861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfcc77-3395-40ba-9b67-2afde54e5888 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.269314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfcc77-3395-40ba-9b67-2afde54e5888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.278371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1111/2217] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:22.279224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfcc77-3395-40ba-9b67-2afde54e5888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:22.280723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfcc77-3395-40ba-9b67-2afde54e5888 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.283381 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938914cf-96f2-4c16-9cc0-30cee0fbd9bc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:22.299131 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938914cf-96f2-4c16-9cc0-30cee0fbd9bc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:22.299352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1107/2218] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:22.307045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bb946-9f46-476f-9b3c-1dd13329b29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:22.310774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb946-9f46-476f-9b3c-1dd13329b29e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.311225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb946-9f46-476f-9b3c-1dd13329b29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.311516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb946-9f46-476f-9b3c-1dd13329b29e admin admin] PATH_INFO: `/v3/projects/c7129d5827e94d7da7452b7803b1b147/users/0592dbfdf4724e179e1be25c121b2c51/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.333816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bb946-9f46-476f-9b3c-1dd13329b29e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7129d5827e94d7da7452b7803b1b147, user_id=0592dbfdf4724e179e1be25c121b2c51, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:22.336727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bb946-9f46-476f-9b3c-1dd13329b29e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.353027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1112/2219] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:22 2022] PUT /identity/v3/projects/c7129d5827e94d7da7452b7803b1b147/users/0592dbfdf4724e179e1be25c121b2c51/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:22.357089 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfcc77-3395-40ba-9b67-2afde54e5888 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:22.368938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfcc77-3395-40ba-9b67-2afde54e5888 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:22.373380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bb946-9f46-476f-9b3c-1dd13329b29e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:22.405284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:22.440474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd81a-ee09-45cd-8d78-939ca3acb3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:22.440860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd81a-ee09-45cd-8d78-939ca3acb3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.440965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd81a-ee09-45cd-8d78-939ca3acb3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.441209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cd81a-ee09-45cd-8d78-939ca3acb3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.441881 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd81a-ee09-45cd-8d78-939ca3acb3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:22.443578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3cd81a-ee09-45cd-8d78-939ca3acb3a5 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.444681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bb946-9f46-476f-9b3c-1dd13329b29e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:22.449660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:22.450015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.450296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.450701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] PATH_INFO: `/v3/users/06c41c2b44c1434a8ee6b26580fe3f42` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.451874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c41c2b44c1434a8ee6b26580fe3f42)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:22.453933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.455144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1108/2220] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:22.459851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd81a-ee09-45cd-8d78-939ca3acb3a5 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:22.470643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cd81a-ee09-45cd-8d78-939ca3acb3a5 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:22.476458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915caad2-8fb1-403a-96dc-a6585ba14386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:22.477125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915caad2-8fb1-403a-96dc-a6585ba14386 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.477734 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915caad2-8fb1-403a-96dc-a6585ba14386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.477859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915caad2-8fb1-403a-96dc-a6585ba14386 admin admin] PATH_INFO: `/v3/projects/c7129d5827e94d7da7452b7803b1b147/users/0592dbfdf4724e179e1be25c121b2c51/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.481179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915caad2-8fb1-403a-96dc-a6585ba14386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7129d5827e94d7da7452b7803b1b147, user_id=0592dbfdf4724e179e1be25c121b2c51, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:22.484560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915caad2-8fb1-403a-96dc-a6585ba14386 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.507969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1109/2221] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:22 2022] PUT /identity/v3/projects/c7129d5827e94d7da7452b7803b1b147/users/0592dbfdf4724e179e1be25c121b2c51/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:22.521045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c41c2b44c1434a8ee6b26580fe3f42'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:22.541446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c41c2b44c1434a8ee6b26580fe3f42'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:22.542824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c41c2b44c1434a8ee6b26580fe3f42'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:22.560039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c41c2b44c1434a8ee6b26580fe3f42'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:22.561470 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c41c2b44c1434a8ee6b26580fe3f42'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:22.587892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1113/2222] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:22 2022] DELETE /identity/v3/users/06c41c2b44c1434a8ee6b26580fe3f42 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:22.589575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f7754-6ce8-4190-b934-3cb8af0d682e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.589866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f7754-6ce8-4190-b934-3cb8af0d682e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.590161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f7754-6ce8-4190-b934-3cb8af0d682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.598676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:22.618753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a73f7754-6ce8-4190-b934-3cb8af0d682e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:22.620152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a73f7754-6ce8-4190-b934-3cb8af0d682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:22.694126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1110/2223] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:22.706500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:22.708486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:22.801710 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e040e-a229-43b4-9e13-7c93c15d46f4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:22.806277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a865b-dcf5-44c9-a725-68ca857ac933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:22.807953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a865b-dcf5-44c9-a725-68ca857ac933 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:22.808183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a865b-dcf5-44c9-a725-68ca857ac933 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:22.808501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a865b-dcf5-44c9-a725-68ca857ac933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:22.822095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a865b-dcf5-44c9-a725-68ca857ac933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:22.823827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a865b-dcf5-44c9-a725-68ca857ac933 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:22.857308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1114/2224] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:23.401211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f7754-6ce8-4190-b934-3cb8af0d682e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:23.403928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a73f7754-6ce8-4190-b934-3cb8af0d682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:23.417651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f7754-6ce8-4190-b934-3cb8af0d682e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:23.423528 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:23.424714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:23.425595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:23.425938 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] PATH_INFO: `/v3/projects/ddbce4474b2b4e9c9e672f76b2bcdc91` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:23.427338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddbce4474b2b4e9c9e672f76b2bcdc91)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:23.430805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:23.504866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddbce4474b2b4e9c9e672f76b2bcdc91', 'request_id': 'req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.512813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddbce4474b2b4e9c9e672f76b2bcdc91', 'request_id': 'req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.522486 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddbce4474b2b4e9c9e672f76b2bcdc91', 'request_id': 'req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.538788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] The token cache is being invalidate because project ddbce4474b2b4e9c9e672f76b2bcdc91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:23.539103 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.540403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.542217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1111/2225] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:23 2022] DELETE /identity/v3/projects/ddbce4474b2b4e9c9e672f76b2bcdc91 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:23.546581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a865b-dcf5-44c9-a725-68ca857ac933 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:23.559165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a865b-dcf5-44c9-a725-68ca857ac933 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:23.622946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:23.645489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:23.646438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:23.646723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:23.647002 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] PATH_INFO: `/v3/users/9b43413d9f3944dd890a3808ffcdd2c6` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:23.648204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b43413d9f3944dd890a3808ffcdd2c6)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:23.650136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:23.731822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b43413d9f3944dd890a3808ffcdd2c6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.748189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b43413d9f3944dd890a3808ffcdd2c6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.748913 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b43413d9f3944dd890a3808ffcdd2c6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.760085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b43413d9f3944dd890a3808ffcdd2c6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.761553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2c45c96b-2809-4a7d-9bb0-0f9b1a1c58a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b43413d9f3944dd890a3808ffcdd2c6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:23.778647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1115/2226] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:23 2022] DELETE /identity/v3/users/9b43413d9f3944dd890a3808ffcdd2c6 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:24.135101 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:24.137290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:24.164783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58b752-4b7b-4810-ab0a-6fec2fd5e875 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:24.242573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:24.243456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.243850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.244369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] PATH_INFO: `/v3/projects/d838d391f03c43138bad70d61f090a78` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:24.245858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d838d391f03c43138bad70d61f090a78)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:24.248353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:24.317726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:24.342030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd838d391f03c43138bad70d61f090a78', 'request_id': 'req-403b999e-4afa-4555-94f1-c612831ac0b4'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:24.349943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd838d391f03c43138bad70d61f090a78', 'request_id': 'req-403b999e-4afa-4555-94f1-c612831ac0b4'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:24.369656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd838d391f03c43138bad70d61f090a78', 'request_id': 'req-403b999e-4afa-4555-94f1-c612831ac0b4'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:24.390204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] The token cache is being invalidate because project d838d391f03c43138bad70d61f090a78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:24.390490 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:24.391597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-403b999e-4afa-4555-94f1-c612831ac0b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:24.398826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1112/2227] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:24 2022] DELETE /identity/v3/projects/d838d391f03c43138bad70d61f090a78 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:24.399249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2914b-9b11-4cc2-a6d8-35c4642ef5f1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.399540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2914b-9b11-4cc2-a6d8-35c4642ef5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.399823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2914b-9b11-4cc2-a6d8-35c4642ef5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:24.425912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-baa2914b-9b11-4cc2-a6d8-35c4642ef5f1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:24.427526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-baa2914b-9b11-4cc2-a6d8-35c4642ef5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:24.538592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1116/2228] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:24.540605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bee31c-de8e-40ec-b2e4-f17be8c8e164 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.541106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bee31c-de8e-40ec-b2e4-f17be8c8e164 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.541295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bee31c-de8e-40ec-b2e4-f17be8c8e164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:24.566588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-b7bee31c-de8e-40ec-b2e4-f17be8c8e164 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:24.568546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b7bee31c-de8e-40ec-b2e4-f17be8c8e164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:24.583537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1113/2229] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:24.587404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2914b-9b11-4cc2-a6d8-35c4642ef5f1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:24.608051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2914b-9b11-4cc2-a6d8-35c4642ef5f1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:24.614626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:24.615312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.615572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.616037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:24.617027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:24.617209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:24.618987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:24.641318 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1117/2230] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:19:24.647846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bee31c-de8e-40ec-b2e4-f17be8c8e164 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:24.659766 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bee31c-de8e-40ec-b2e4-f17be8c8e164 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:24.666247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafed59-060a-4f4a-bf9b-1efd07a428b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:24.666868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafed59-060a-4f4a-bf9b-1efd07a428b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.667180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafed59-060a-4f4a-bf9b-1efd07a428b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.667452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafed59-060a-4f4a-bf9b-1efd07a428b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:24.668100 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafed59-060a-4f4a-bf9b-1efd07a428b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:24.670621 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafed59-060a-4f4a-bf9b-1efd07a428b6 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:24.717541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1114/2231] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:24 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:24.722520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:24.738898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912adb93-bc44-4792-a57f-84d9d2b25a78 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:24.836592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:24.850454 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2df92b-e3e9-4e3f-ab19-4b8bd1fbaea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:24.851321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2df92b-e3e9-4e3f-ab19-4b8bd1fbaea7 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.851900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2df92b-e3e9-4e3f-ab19-4b8bd1fbaea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.852283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2df92b-e3e9-4e3f-ab19-4b8bd1fbaea7 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:24.853268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2df92b-e3e9-4e3f-ab19-4b8bd1fbaea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:24.855180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2df92b-e3e9-4e3f-ab19-4b8bd1fbaea7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:24.886611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1118/2232] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:24 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:24.890771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafed59-060a-4f4a-bf9b-1efd07a428b6 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:24.901985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafed59-060a-4f4a-bf9b-1efd07a428b6 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:24.907006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c27e80-f325-400d-9030-ec16aecd95ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:24.907672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27e80-f325-400d-9030-ec16aecd95ab admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.907950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27e80-f325-400d-9030-ec16aecd95ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.909215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c27e80-f325-400d-9030-ec16aecd95ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:24.909215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c27e80-f325-400d-9030-ec16aecd95ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:24.910789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c27e80-f325-400d-9030-ec16aecd95ab admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:24.926355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1115/2233] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:24.928101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2df92b-e3e9-4e3f-ab19-4b8bd1fbaea7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:24.939481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2df92b-e3e9-4e3f-ab19-4b8bd1fbaea7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:24.946041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498e1207-0877-497c-a0a1-008d941f6193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:24.946733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1207-0877-497c-a0a1-008d941f6193 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.947248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1207-0877-497c-a0a1-008d941f6193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.947574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498e1207-0877-497c-a0a1-008d941f6193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:24.948341 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e1207-0877-497c-a0a1-008d941f6193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:24.950038 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498e1207-0877-497c-a0a1-008d941f6193 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:24.963339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1119/2234] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:24.967752 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c27e80-f325-400d-9030-ec16aecd95ab admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:24.979014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c27e80-f325-400d-9030-ec16aecd95ab admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:24.984010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff8dd6-0902-4107-9e2e-2e7e29df84dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:24.984739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff8dd6-0902-4107-9e2e-2e7e29df84dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:24.985087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff8dd6-0902-4107-9e2e-2e7e29df84dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:24.985430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff8dd6-0902-4107-9e2e-2e7e29df84dc admin admin] PATH_INFO: `/v3/projects/a5a51e54d19a4a0da468a57cdf3974ab/users/0e482c4c6fe94d8da471826d1927929d/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.004825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff8dd6-0902-4107-9e2e-2e7e29df84dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a51e54d19a4a0da468a57cdf3974ab, user_id=0e482c4c6fe94d8da471826d1927929d, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:25.008205 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff8dd6-0902-4107-9e2e-2e7e29df84dc admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:25.023286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1116/2235] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:24 2022] PUT /identity/v3/projects/a5a51e54d19a4a0da468a57cdf3974ab/users/0e482c4c6fe94d8da471826d1927929d/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:25.034708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41596215-7754-4b18-b6bb-b6743237786a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.035471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41596215-7754-4b18-b6bb-b6743237786a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.035998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41596215-7754-4b18-b6bb-b6743237786a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.087312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-41596215-7754-4b18-b6bb-b6743237786a None None] MFA Rules not processed for user `0e482c4c6fe94d8da471826d1927929d`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:25.088535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-41596215-7754-4b18-b6bb-b6743237786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:25.187219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1120/2236] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:25.192515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff8dd6-0902-4107-9e2e-2e7e29df84dc admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:25.300050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff8dd6-0902-4107-9e2e-2e7e29df84dc admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:25.305782 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c748b0-622a-44c4-ba91-e6c48164126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:25.306656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c748b0-622a-44c4-ba91-e6c48164126f admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.306910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c748b0-622a-44c4-ba91-e6c48164126f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.307217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c748b0-622a-44c4-ba91-e6c48164126f admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.307972 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c748b0-622a-44c4-ba91-e6c48164126f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:25.310035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c748b0-622a-44c4-ba91-e6c48164126f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:25.360944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1117/2237] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:25 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:25.364957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41596215-7754-4b18-b6bb-b6743237786a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:25.377474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41596215-7754-4b18-b6bb-b6743237786a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:25.509802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec664a-67c6-4f77-871f-7fa351f09261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:25.511300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec664a-67c6-4f77-871f-7fa351f09261 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.511666 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec664a-67c6-4f77-871f-7fa351f09261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.512015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec664a-67c6-4f77-871f-7fa351f09261 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.512735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec664a-67c6-4f77-871f-7fa351f09261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:25.514923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec664a-67c6-4f77-871f-7fa351f09261 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:25.564562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1121/2238] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:25 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:25.567719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c748b0-622a-44c4-ba91-e6c48164126f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:25.583756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c748b0-622a-44c4-ba91-e6c48164126f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:25.597969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ed5ae-53cf-4811-8b9f-dd303efb1d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:25.598783 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed5ae-53cf-4811-8b9f-dd303efb1d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.601262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed5ae-53cf-4811-8b9f-dd303efb1d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.605010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185ed5ae-53cf-4811-8b9f-dd303efb1d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.606099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ed5ae-53cf-4811-8b9f-dd303efb1d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:25.610191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185ed5ae-53cf-4811-8b9f-dd303efb1d5e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:25.626008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1118/2239] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:25.630738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec664a-67c6-4f77-871f-7fa351f09261 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:25.645970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec664a-67c6-4f77-871f-7fa351f09261 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:25.652770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32f15f-d177-4b15-900d-a7153f03cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:25.653645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f15f-d177-4b15-900d-a7153f03cd4e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.653998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f15f-d177-4b15-900d-a7153f03cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.654347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f15f-d177-4b15-900d-a7153f03cd4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.655191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32f15f-d177-4b15-900d-a7153f03cd4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:25.657022 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32f15f-d177-4b15-900d-a7153f03cd4e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:25.676327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ed5ae-53cf-4811-8b9f-dd303efb1d5e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:25.677444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1122/2240] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:25.687586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185ed5ae-53cf-4811-8b9f-dd303efb1d5e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:25.693944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05813d-29d2-4604-a538-3a840f388908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:25.694741 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05813d-29d2-4604-a538-3a840f388908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.695181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05813d-29d2-4604-a538-3a840f388908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.695593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05813d-29d2-4604-a538-3a840f388908 admin admin] PATH_INFO: `/v3/projects/2df87f716be64905891d6439ed180496/users/994af5f214df4b75a571ca08a8849077/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.717391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05813d-29d2-4604-a538-3a840f388908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df87f716be64905891d6439ed180496, user_id=994af5f214df4b75a571ca08a8849077, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:25.721342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05813d-29d2-4604-a538-3a840f388908 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:25.740654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1119/2241] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:25 2022] PUT /identity/v3/projects/2df87f716be64905891d6439ed180496/users/994af5f214df4b75a571ca08a8849077/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:25.751463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c82c-aca8-47f7-899b-afd310acba80 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:25.751763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c82c-aca8-47f7-899b-afd310acba80 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:25.752047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057c82c-aca8-47f7-899b-afd310acba80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:25.793792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5057c82c-aca8-47f7-899b-afd310acba80 None None] MFA Rules not processed for user `994af5f214df4b75a571ca08a8849077`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:25.795259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5057c82c-aca8-47f7-899b-afd310acba80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:25.838056 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:25.896137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1123/2242] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:25.900801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05813d-29d2-4604-a538-3a840f388908 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:25.908690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:26.048981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05813d-29d2-4604-a538-3a840f388908 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:26.061585 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adcf99-c795-47ba-808a-5ef92708a8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:26.068390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adcf99-c795-47ba-808a-5ef92708a8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:26.068779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adcf99-c795-47ba-808a-5ef92708a8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:26.069156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adcf99-c795-47ba-808a-5ef92708a8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.069936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adcf99-c795-47ba-808a-5ef92708a8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:26.080484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adcf99-c795-47ba-808a-5ef92708a8ac admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:26.144739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1120/2243] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:25 2022] POST /identity/v3/projects => generated 387 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:26.145625 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057c82c-aca8-47f7-899b-afd310acba80 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:26.195920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057c82c-aca8-47f7-899b-afd310acba80 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:26.342575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eca6ed-62c9-4ca3-a166-8cd263d20eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:26.343614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eca6ed-62c9-4ca3-a166-8cd263d20eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:26.344003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eca6ed-62c9-4ca3-a166-8cd263d20eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:26.344425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eca6ed-62c9-4ca3-a166-8cd263d20eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.345271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eca6ed-62c9-4ca3-a166-8cd263d20eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:26.348561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eca6ed-62c9-4ca3-a166-8cd263d20eaf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:26.395966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1124/2244] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:26 2022] POST /identity/v3/users => generated 300 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:26.400058 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adcf99-c795-47ba-808a-5ef92708a8ac admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:26.415646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adcf99-c795-47ba-808a-5ef92708a8ac admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:26.426003 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629c1c2-f645-45f0-b27f-f7485a1797b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:26.426806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629c1c2-f645-45f0-b27f-f7485a1797b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:26.427253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629c1c2-f645-45f0-b27f-f7485a1797b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:26.427633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629c1c2-f645-45f0-b27f-f7485a1797b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.428537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629c1c2-f645-45f0-b27f-f7485a1797b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:26.433446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629c1c2-f645-45f0-b27f-f7485a1797b6 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:26.466818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eca6ed-62c9-4ca3-a166-8cd263d20eaf admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:26.474406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1121/2245] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:26.484218 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eca6ed-62c9-4ca3-a166-8cd263d20eaf admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:26.502338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a8479-9e5e-41fc-8244-3c0253ee0b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:26.503414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a8479-9e5e-41fc-8244-3c0253ee0b29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:26.503954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a8479-9e5e-41fc-8244-3c0253ee0b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:26.504393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a8479-9e5e-41fc-8244-3c0253ee0b29 admin admin] PATH_INFO: `/v3/projects/f1834fd4f28b44a6a4700fe740aed5aa/users/b1d9be69943e4fc19e0547cf8c57eef3/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.552842 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a8479-9e5e-41fc-8244-3c0253ee0b29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1834fd4f28b44a6a4700fe740aed5aa, user_id=b1d9be69943e4fc19e0547cf8c57eef3, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:26.561967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2a8479-9e5e-41fc-8244-3c0253ee0b29 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:26.603905 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629c1c2-f645-45f0-b27f-f7485a1797b6 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:26.604463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1125/2246] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:26 2022] PUT /identity/v3/projects/f1834fd4f28b44a6a4700fe740aed5aa/users/b1d9be69943e4fc19e0547cf8c57eef3/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:26.617771 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629c1c2-f645-45f0-b27f-f7485a1797b6 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:26.759289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cda6dc-3e29-4797-a86d-82bcc1f38375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:26.760023 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda6dc-3e29-4797-a86d-82bcc1f38375 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:26.760409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda6dc-3e29-4797-a86d-82bcc1f38375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:26.761004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cda6dc-3e29-4797-a86d-82bcc1f38375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.761999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cda6dc-3e29-4797-a86d-82bcc1f38375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:26.763788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cda6dc-3e29-4797-a86d-82bcc1f38375 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:26.793985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a8479-9e5e-41fc-8244-3c0253ee0b29 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:26.804979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1122/2247] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:26.809303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a8479-9e5e-41fc-8244-3c0253ee0b29 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:26.816199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:26.817210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:26.817702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:26.818301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] PATH_INFO: `/v3/projects/f1834fd4f28b44a6a4700fe740aed5aa/users/b1d9be69943e4fc19e0547cf8c57eef3/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.822160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1834fd4f28b44a6a4700fe740aed5aa, user_id=b1d9be69943e4fc19e0547cf8c57eef3, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:26.825352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:26.856400 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6107f-7ec0-4b99-8af2-1527c4319e43 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:26.856693 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6107f-7ec0-4b99-8af2-1527c4319e43 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:26.856967 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6107f-7ec0-4b99-8af2-1527c4319e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:26.863614 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1126/2248] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:26 2022] PUT /identity/v3/projects/f1834fd4f28b44a6a4700fe740aed5aa/users/b1d9be69943e4fc19e0547cf8c57eef3/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:26.898757 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-74e6107f-7ec0-4b99-8af2-1527c4319e43 None None] MFA Rules not processed for user `b1d9be69943e4fc19e0547cf8c57eef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:26.900292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-74e6107f-7ec0-4b99-8af2-1527c4319e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:27.036948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:27.059436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1123/2249] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:27.083152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:27.094065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:27.234863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583f3d5-99f3-4b4c-b494-9ba9de9ffe9e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:27.239006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd9966-6f15-4e03-919e-fe775734ec46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:27.239517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9966-6f15-4e03-919e-fe775734ec46 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:27.239802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9966-6f15-4e03-919e-fe775734ec46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:27.240098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd9966-6f15-4e03-919e-fe775734ec46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:27.256278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd9966-6f15-4e03-919e-fe775734ec46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:27.257953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd9966-6f15-4e03-919e-fe775734ec46 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:27.288977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1127/2250] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:28.143909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6107f-7ec0-4b99-8af2-1527c4319e43 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:28.148875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-74e6107f-7ec0-4b99-8af2-1527c4319e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:28.168926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6107f-7ec0-4b99-8af2-1527c4319e43 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:28.172998 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:28.173697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:28.174040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:28.174388 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:28.336904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:28.339777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:28.354784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1124/2251] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:19:28.459733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12433eaf-cdc0-4a74-9750-2d2add7b4e3c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:28.461140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12433eaf-cdc0-4a74-9750-2d2add7b4e3c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:28.461445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12433eaf-cdc0-4a74-9750-2d2add7b4e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:28.535623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-12433eaf-cdc0-4a74-9750-2d2add7b4e3c None None] MFA Rules not processed for user `0592dbfdf4724e179e1be25c121b2c51`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:28.537236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-12433eaf-cdc0-4a74-9750-2d2add7b4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:28.563455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:28.681623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1128/2252] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:28.696155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:28.698271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:28.868178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b3c30-e555-4b1d-803f-57ceeaf16d6a service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:28.883347 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:28.884399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:28.884878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:28.885178 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:28.887295 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:28.967802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:28.969742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:28.981398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1125/2253] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:30.579840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352126f-5392-4dab-ba01-1a7ca384988a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:30.580307 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352126f-5392-4dab-ba01-1a7ca384988a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:30.580469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352126f-5392-4dab-ba01-1a7ca384988a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:30.600801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9352126f-5392-4dab-ba01-1a7ca384988a None None] MFA Rules not processed for user `af98e8f344914ae59852d82d176b4dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:30.602033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9352126f-5392-4dab-ba01-1a7ca384988a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:30.609542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1129/2254] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:32.929875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:32.972423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75206bde-f291-4737-9d99-444dd9c8863e service nova] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:32.976581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:32.977301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:32.977688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:32.978059 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:33.021401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:33.061323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:33.064760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:33.075176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1126/2255] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:35.257482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352126f-5392-4dab-ba01-1a7ca384988a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:35.324033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352126f-5392-4dab-ba01-1a7ca384988a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:35.328851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:35.329638 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:35.329924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:35.330282 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] PATH_INFO: `/v3/users/0e482c4c6fe94d8da471826d1927929d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:35.331728 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e482c4c6fe94d8da471826d1927929d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:35.333721 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:35.401991 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e482c4c6fe94d8da471826d1927929d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:35.419164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e482c4c6fe94d8da471826d1927929d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:35.419939 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e482c4c6fe94d8da471826d1927929d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:35.428669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e482c4c6fe94d8da471826d1927929d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:35.429293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3953ec27-8e88-4477-8f8d-db9f1b48f3bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e482c4c6fe94d8da471826d1927929d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:35.445482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:35.446463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1130/2256] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:35 2022] DELETE /identity/v3/users/0e482c4c6fe94d8da471826d1927929d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:35.446996 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:35.450720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf91cd-cf25-4f1e-a096-0663c2fb7299 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:35.451014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf91cd-cf25-4f1e-a096-0663c2fb7299 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:35.451330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf91cd-cf25-4f1e-a096-0663c2fb7299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:35.474342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dddf91cd-cf25-4f1e-a096-0663c2fb7299 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:35.475612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dddf91cd-cf25-4f1e-a096-0663c2fb7299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:35.496206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:35.536847 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9a92a-88b7-42a5-9154-7f74fb6ed16b service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:35.554283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1131/2257] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:35.561073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf91cd-cf25-4f1e-a096-0663c2fb7299 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:35.562566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dddf91cd-cf25-4f1e-a096-0663c2fb7299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:35.593061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:35.593578 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:35.593857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:35.594129 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:35.595911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:35.609027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:35.610457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:35.624430 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf91cd-cf25-4f1e-a096-0663c2fb7299 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:35.625697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1127/2258] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:19:35.628401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22774d24-cc1f-4a5e-a181-eb3f9025ff62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:35.629156 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774d24-cc1f-4a5e-a181-eb3f9025ff62 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:35.629458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774d24-cc1f-4a5e-a181-eb3f9025ff62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:35.629729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774d24-cc1f-4a5e-a181-eb3f9025ff62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:35.639917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22774d24-cc1f-4a5e-a181-eb3f9025ff62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:35.641362 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22774d24-cc1f-4a5e-a181-eb3f9025ff62 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:35.651140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1132/2259] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:19:36.387984 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:36.389441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:36.407721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd62f78-7d29-4f72-974f-4a9923bfb298 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:36.416189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:36.416845 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:36.417141 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:36.417372 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] PATH_INFO: `/v3/projects/a5a51e54d19a4a0da468a57cdf3974ab` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:36.418708 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a51e54d19a4a0da468a57cdf3974ab)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:36.420527 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:36.495858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a51e54d19a4a0da468a57cdf3974ab', 'request_id': 'req-579c9bc2-5da5-4b25-b952-8387ac74240a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.502740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a51e54d19a4a0da468a57cdf3974ab', 'request_id': 'req-579c9bc2-5da5-4b25-b952-8387ac74240a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.513360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a51e54d19a4a0da468a57cdf3974ab', 'request_id': 'req-579c9bc2-5da5-4b25-b952-8387ac74240a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.527882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] The token cache is being invalidate because project a5a51e54d19a4a0da468a57cdf3974ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:36.528170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.528839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.536831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1128/2260] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:36 2022] DELETE /identity/v3/projects/a5a51e54d19a4a0da468a57cdf3974ab => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:36.541215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22774d24-cc1f-4a5e-a181-eb3f9025ff62 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:36.551247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22774d24-cc1f-4a5e-a181-eb3f9025ff62 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:36.635996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:36.636652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:36.636933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:36.637210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] PATH_INFO: `/v3/users/994af5f214df4b75a571ca08a8849077` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:36.638349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994af5f214df4b75a571ca08a8849077)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:36.640344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:36.702989 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994af5f214df4b75a571ca08a8849077'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.718171 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994af5f214df4b75a571ca08a8849077'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.718485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994af5f214df4b75a571ca08a8849077'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.725283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:36.730834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994af5f214df4b75a571ca08a8849077'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.734542 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994af5f214df4b75a571ca08a8849077'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:36.755328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1133/2261] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:36 2022] DELETE /identity/v3/users/994af5f214df4b75a571ca08a8849077 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:37.242870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:37.244360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:37.262232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c9bc2-5da5-4b25-b952-8387ac74240a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:37.286816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:37.353421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:37.354165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:37.354518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:37.354890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] PATH_INFO: `/v3/projects/2df87f716be64905891d6439ed180496` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:37.356140 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df87f716be64905891d6439ed180496)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:37.358125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:37.416411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df87f716be64905891d6439ed180496', 'request_id': 'req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.424145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df87f716be64905891d6439ed180496', 'request_id': 'req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.436256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df87f716be64905891d6439ed180496', 'request_id': 'req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.452016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] The token cache is being invalidate because project 2df87f716be64905891d6439ed180496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:37.453107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.454017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.456722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1129/2262] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:37 2022] DELETE /identity/v3/projects/2df87f716be64905891d6439ed180496 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:37.460540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:37.462012 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:37.476211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f6a9f-ff3a-4c57-aff0-fe7932bab8e9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:37.543692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:37.544343 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:37.544779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:37.545060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] PATH_INFO: `/v3/users/b1d9be69943e4fc19e0547cf8c57eef3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:37.546189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d9be69943e4fc19e0547cf8c57eef3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:37.548169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:37.611839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d9be69943e4fc19e0547cf8c57eef3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.628661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d9be69943e4fc19e0547cf8c57eef3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.630143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d9be69943e4fc19e0547cf8c57eef3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.640172 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d9be69943e4fc19e0547cf8c57eef3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.641994 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a47934b2-bedf-49a3-9b9c-091665483025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d9be69943e4fc19e0547cf8c57eef3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:37.663485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1134/2263] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:37 2022] DELETE /identity/v3/users/b1d9be69943e4fc19e0547cf8c57eef3 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:38.577700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:38.579406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:38.600044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739bdd11-d6c0-4f90-9f3e-b07b5a9b0e2d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:38.728577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:38.729496 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:38.729948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:38.730396 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] PATH_INFO: `/v3/projects/f1834fd4f28b44a6a4700fe740aed5aa` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:38.732398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1834fd4f28b44a6a4700fe740aed5aa)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:38.734334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:38.796027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:38.798945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1834fd4f28b44a6a4700fe740aed5aa', 'request_id': 'req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:38.812649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1834fd4f28b44a6a4700fe740aed5aa', 'request_id': 'req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:38.831326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1834fd4f28b44a6a4700fe740aed5aa', 'request_id': 'req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:38.845533 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] The token cache is being invalidate because project f1834fd4f28b44a6a4700fe740aed5aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:38.845908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:38.847405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9b6aa998-86f9-4f65-81b5-cfb260fb6f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:38.856157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1130/2264] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:38 2022] DELETE /identity/v3/projects/f1834fd4f28b44a6a4700fe740aed5aa => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:38.857352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4374-cbd0-4710-803d-bffbc801293d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:38.857808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4374-cbd0-4710-803d-bffbc801293d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:38.858514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b4374-cbd0-4710-803d-bffbc801293d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:38.884380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f47b4374-cbd0-4710-803d-bffbc801293d None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:38.886996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f47b4374-cbd0-4710-803d-bffbc801293d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:38.979112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1135/2265] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:38.987458 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5890c0-ed85-44fc-b189-19de72d5d8df None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:38.988213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5890c0-ed85-44fc-b189-19de72d5d8df None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:38.988598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5890c0-ed85-44fc-b189-19de72d5d8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:39.013956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-ee5890c0-ed85-44fc-b189-19de72d5d8df None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:39.015385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ee5890c0-ed85-44fc-b189-19de72d5d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:39.024853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1131/2266] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:39.029183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b4374-cbd0-4710-803d-bffbc801293d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:39.041404 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b4374-cbd0-4710-803d-bffbc801293d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:39.047245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:39.047891 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:39.048329 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:39.048897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:39.050164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:39.050476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:39.052289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:39.054448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:39.071264 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1136/2267] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:19:39.075907 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5890c0-ed85-44fc-b189-19de72d5d8df None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:39.092562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5890c0-ed85-44fc-b189-19de72d5d8df None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:39.098549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae87fa-52cf-4f13-90e2-7ff2947fedef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:39.099356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae87fa-52cf-4f13-90e2-7ff2947fedef admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:39.099868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae87fa-52cf-4f13-90e2-7ff2947fedef admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:39.100150 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efae87fa-52cf-4f13-90e2-7ff2947fedef admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:39.100890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae87fa-52cf-4f13-90e2-7ff2947fedef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:39.102989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efae87fa-52cf-4f13-90e2-7ff2947fedef admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:39.149181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1132/2268] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:39 2022] POST /identity/v3/projects => generated 361 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:39.150677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:39.161992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aaf955-3ee5-4401-a9ba-e8a571d779f8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:39.254870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21599e-24a9-4265-a120-4ff510dc5e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:39.255565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21599e-24a9-4265-a120-4ff510dc5e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:39.255853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21599e-24a9-4265-a120-4ff510dc5e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:39.256135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21599e-24a9-4265-a120-4ff510dc5e1f admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:39.256791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21599e-24a9-4265-a120-4ff510dc5e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:39.258872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21599e-24a9-4265-a120-4ff510dc5e1f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:39.293249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1137/2269] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:39 2022] POST /identity/v3/users => generated 281 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:39.297227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae87fa-52cf-4f13-90e2-7ff2947fedef admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:39.319611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efae87fa-52cf-4f13-90e2-7ff2947fedef admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:39.332195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f41c86-6ebc-4d61-86f6-0762c492e70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:39.332835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41c86-6ebc-4d61-86f6-0762c492e70f admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:39.333121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41c86-6ebc-4d61-86f6-0762c492e70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:39.333402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f41c86-6ebc-4d61-86f6-0762c492e70f admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:39.334066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f41c86-6ebc-4d61-86f6-0762c492e70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:39.336556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f41c86-6ebc-4d61-86f6-0762c492e70f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:39.358795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1133/2270] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:39.359589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21599e-24a9-4265-a120-4ff510dc5e1f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:39.375535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21599e-24a9-4265-a120-4ff510dc5e1f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:39.386011 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d42160-65ae-4c82-ac62-72c2fbde3730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:39.386746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42160-65ae-4c82-ac62-72c2fbde3730 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:39.387033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42160-65ae-4c82-ac62-72c2fbde3730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:39.387460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d42160-65ae-4c82-ac62-72c2fbde3730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:39.388015 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d42160-65ae-4c82-ac62-72c2fbde3730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:39.389997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d42160-65ae-4c82-ac62-72c2fbde3730 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:39.404338 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1138/2271] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:39.408697 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f41c86-6ebc-4d61-86f6-0762c492e70f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:39.422179 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f41c86-6ebc-4d61-86f6-0762c492e70f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:39.429618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:39.430953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:39.431293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:39.431576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] PATH_INFO: `/v3/projects/15080cc0eff94dd2a7e8936d8ca9a52b/users/b931b05896ed42ab82e7f96e537cf29a/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:39.475369 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] RBAC: Authorizing `identity:create_grant(project_id=15080cc0eff94dd2a7e8936d8ca9a52b, user_id=b931b05896ed42ab82e7f96e537cf29a, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:39.480284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:39.506327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1134/2272] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:39 2022] PUT /identity/v3/projects/15080cc0eff94dd2a7e8936d8ca9a52b/users/b931b05896ed42ab82e7f96e537cf29a/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:39.521866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47c19-0407-47a4-946e-aac7fbe5056e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:39.522375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47c19-0407-47a4-946e-aac7fbe5056e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:39.522840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d47c19-0407-47a4-946e-aac7fbe5056e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:39.572027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-22d47c19-0407-47a4-946e-aac7fbe5056e None None] MFA Rules not processed for user `b931b05896ed42ab82e7f96e537cf29a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:39.573371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-22d47c19-0407-47a4-946e-aac7fbe5056e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:39.716235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1139/2273] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:39.763105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:39.767991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:40.007434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5d0ee-94bc-4b23-8f1a-9bfab012900f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:40.011427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520e7bb-3923-4b9c-8edc-53326f668599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:40.012055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e7bb-3923-4b9c-8edc-53326f668599 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:40.012335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e7bb-3923-4b9c-8edc-53326f668599 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:40.012630 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520e7bb-3923-4b9c-8edc-53326f668599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:40.025210 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520e7bb-3923-4b9c-8edc-53326f668599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:40.026657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520e7bb-3923-4b9c-8edc-53326f668599 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:40.050430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1135/2274] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:40.066876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d47c19-0407-47a4-946e-aac7fbe5056e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:40.164724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d47c19-0407-47a4-946e-aac7fbe5056e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:40.170568 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:40.171244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:40.171546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:40.171811 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] PATH_INFO: `/v3/users/b931b05896ed42ab82e7f96e537cf29a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:40.172947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b931b05896ed42ab82e7f96e537cf29a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:40.175686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:40.178919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:40.233738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b931b05896ed42ab82e7f96e537cf29a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.250667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b931b05896ed42ab82e7f96e537cf29a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.251791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b931b05896ed42ab82e7f96e537cf29a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.272787 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b931b05896ed42ab82e7f96e537cf29a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.274085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b931b05896ed42ab82e7f96e537cf29a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.313243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1140/2275] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:40 2022] DELETE /identity/v3/users/b931b05896ed42ab82e7f96e537cf29a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:40.318404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769c027-ce51-4439-90f4-7c279d186d29 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:40.323573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769c027-ce51-4439-90f4-7c279d186d29 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:40.323864 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769c027-ce51-4439-90f4-7c279d186d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:40.361197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5769c027-ce51-4439-90f4-7c279d186d29 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:40.363549 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5769c027-ce51-4439-90f4-7c279d186d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:40.475557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:40.477368 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:40.512012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1136/2276] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:40.523718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769c027-ce51-4439-90f4-7c279d186d29 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:40.530394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5769c027-ce51-4439-90f4-7c279d186d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:40.550301 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:40.596713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1c6c6-06ee-459f-8ba4-5b54670a7dd8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:40.601791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:40.602512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:40.602814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:40.603096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] PATH_INFO: `/v3/users/3f69df43e6ca4663a9ef819c0331c13d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:40.604254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f69df43e6ca4663a9ef819c0331c13d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:40.606265 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:40.625186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769c027-ce51-4439-90f4-7c279d186d29 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:40.629087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36303f74-1e63-4fa0-83ec-7b56663f566e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:40.629695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36303f74-1e63-4fa0-83ec-7b56663f566e service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:40.629978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36303f74-1e63-4fa0-83ec-7b56663f566e service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:40.630281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36303f74-1e63-4fa0-83ec-7b56663f566e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:40.641676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36303f74-1e63-4fa0-83ec-7b56663f566e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:40.643296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36303f74-1e63-4fa0-83ec-7b56663f566e service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:40.663279 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1137/2277] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:19:40.673832 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f69df43e6ca4663a9ef819c0331c13d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.703961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f69df43e6ca4663a9ef819c0331c13d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.704965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f69df43e6ca4663a9ef819c0331c13d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.723120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f69df43e6ca4663a9ef819c0331c13d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.724346 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f69df43e6ca4663a9ef819c0331c13d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:40.740587 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:40.754060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1141/2278] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:40 2022] DELETE /identity/v3/users/3f69df43e6ca4663a9ef819c0331c13d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:41.044119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36303f74-1e63-4fa0-83ec-7b56663f566e service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.064516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36303f74-1e63-4fa0-83ec-7b56663f566e service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.189602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.193429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:41.210454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.212208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.215032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.215472 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] PATH_INFO: `/v3/projects/df0bdf3ab29d43e0bf749b9814813e5c` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.218624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0bdf3ab29d43e0bf749b9814813e5c)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.226424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.230437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c7ef5-f527-4e66-8607-16b12f74ba59 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.241121 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.241979 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.242514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.243444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] PATH_INFO: `/v3/projects/15080cc0eff94dd2a7e8936d8ca9a52b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.245432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15080cc0eff94dd2a7e8936d8ca9a52b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.249084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.301488 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0bdf3ab29d43e0bf749b9814813e5c', 'request_id': 'req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.310460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0bdf3ab29d43e0bf749b9814813e5c', 'request_id': 'req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.313653 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15080cc0eff94dd2a7e8936d8ca9a52b', 'request_id': 'req-b2d3d47d-395e-49ae-8998-b635eb4872c7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.325537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15080cc0eff94dd2a7e8936d8ca9a52b', 'request_id': 'req-b2d3d47d-395e-49ae-8998-b635eb4872c7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.326164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0bdf3ab29d43e0bf749b9814813e5c', 'request_id': 'req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.337982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15080cc0eff94dd2a7e8936d8ca9a52b', 'request_id': 'req-b2d3d47d-395e-49ae-8998-b635eb4872c7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.344860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] The token cache is being invalidate because project df0bdf3ab29d43e0bf749b9814813e5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:41.345443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.346408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-d4bc5ff9-17f3-47c4-bacd-fc38aeb6a13d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.352128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] The token cache is being invalidate because project 15080cc0eff94dd2a7e8936d8ca9a52b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:41.352128 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.352691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1138/2279] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:41 2022] DELETE /identity/v3/projects/df0bdf3ab29d43e0bf749b9814813e5c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:41.352857 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b2d3d47d-395e-49ae-8998-b635eb4872c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:41.354464 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1142/2280] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:41 2022] DELETE /identity/v3/projects/15080cc0eff94dd2a7e8936d8ca9a52b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:41.354754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e3e3a-bfe1-4b69-a9c6-8b1e8e514f39 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.355345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e3e3a-bfe1-4b69-a9c6-8b1e8e514f39 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.355760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e3e3a-bfe1-4b69-a9c6-8b1e8e514f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.360087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b874-cf11-444b-9e41-14d17559145b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.360495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b874-cf11-444b-9e41-14d17559145b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.360875 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b874-cf11-444b-9e41-14d17559145b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.381111 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-091e3e3a-bfe1-4b69-a9c6-8b1e8e514f39 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:41.382381 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-39f6b874-cf11-444b-9e41-14d17559145b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:41.383283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-091e3e3a-bfe1-4b69-a9c6-8b1e8e514f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:41.384186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-39f6b874-cf11-444b-9e41-14d17559145b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:41.453759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:41.487828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1143/2281] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:41.497547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470a96c-8029-4e78-8c4f-4e0ba49288c9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.497547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470a96c-8029-4e78-8c4f-4e0ba49288c9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.497881 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1139/2282] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:41.497998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470a96c-8029-4e78-8c4f-4e0ba49288c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.500673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c2b24-2458-4e28-8eaa-0692f636bdfc None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.501073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c2b24-2458-4e28-8eaa-0692f636bdfc None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.501457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c2b24-2458-4e28-8eaa-0692f636bdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.528802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-351c2b24-2458-4e28-8eaa-0692f636bdfc None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:41.530952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-351c2b24-2458-4e28-8eaa-0692f636bdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:41.534961 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e470a96c-8029-4e78-8c4f-4e0ba49288c9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:41.536185 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e470a96c-8029-4e78-8c4f-4e0ba49288c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:41.541132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1140/2283] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:41.545315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c2b24-2458-4e28-8eaa-0692f636bdfc None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.552982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1144/2284] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:41.558281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470a96c-8029-4e78-8c4f-4e0ba49288c9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.561923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c2b24-2458-4e28-8eaa-0692f636bdfc None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.567394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.568090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.568442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.568804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.569757 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:41.570130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.572678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.576062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470a96c-8029-4e78-8c4f-4e0ba49288c9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.581839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.582493 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.582747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.583082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.583763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:41.584168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.586148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.587123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1141/2285] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:19:41.591786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.603456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb29a7-feac-40e0-9d2b-d3a237266bf6 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.605352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1145/2286] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:19:41.610299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.614074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65228ece-34e0-4c89-89ae-9820c4b3ac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.615333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65228ece-34e0-4c89-89ae-9820c4b3ac48 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.615685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65228ece-34e0-4c89-89ae-9820c4b3ac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.616356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65228ece-34e0-4c89-89ae-9820c4b3ac48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.616855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65228ece-34e0-4c89-89ae-9820c4b3ac48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.619755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65228ece-34e0-4c89-89ae-9820c4b3ac48 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.659226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e71e88a-5047-4a5a-81e3-b34f4adc97c0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.671401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1142/2287] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:41 2022] POST /identity/v3/projects => generated 357 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:41.672644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65228ece-34e0-4c89-89ae-9820c4b3ac48 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.734627 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae8e06-e87d-492a-8ee7-c22c72dc45d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.735885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae8e06-e87d-492a-8ee7-c22c72dc45d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.736380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae8e06-e87d-492a-8ee7-c22c72dc45d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.736521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae8e06-e87d-492a-8ee7-c22c72dc45d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.737672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae8e06-e87d-492a-8ee7-c22c72dc45d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.739617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae8e06-e87d-492a-8ee7-c22c72dc45d5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.755417 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:41.762898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65228ece-34e0-4c89-89ae-9820c4b3ac48 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.769624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107619a2-d275-42b2-b47c-708b269dfa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.770299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107619a2-d275-42b2-b47c-708b269dfa5c admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.770622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107619a2-d275-42b2-b47c-708b269dfa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.770860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107619a2-d275-42b2-b47c-708b269dfa5c admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.771556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107619a2-d275-42b2-b47c-708b269dfa5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.773514 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107619a2-d275-42b2-b47c-708b269dfa5c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.775115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1146/2288] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:41 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:41.778936 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae8e06-e87d-492a-8ee7-c22c72dc45d5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.802817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1143/2289] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:41 2022] POST /identity/v3/users => generated 279 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:41.805858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107619a2-d275-42b2-b47c-708b269dfa5c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.823816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107619a2-d275-42b2-b47c-708b269dfa5c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.865379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae8e06-e87d-492a-8ee7-c22c72dc45d5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.874624 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc83d-ee2f-44a6-bcbf-46251e089541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.875342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc83d-ee2f-44a6-bcbf-46251e089541 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.875771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc83d-ee2f-44a6-bcbf-46251e089541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.876091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfc83d-ee2f-44a6-bcbf-46251e089541 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.876835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfc83d-ee2f-44a6-bcbf-46251e089541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.879658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfc83d-ee2f-44a6-bcbf-46251e089541 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.895959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eec989-72e8-4af7-bd0e-70a683fb740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.896842 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eec989-72e8-4af7-bd0e-70a683fb740b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.897121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eec989-72e8-4af7-bd0e-70a683fb740b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.897694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eec989-72e8-4af7-bd0e-70a683fb740b admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.898346 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eec989-72e8-4af7-bd0e-70a683fb740b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.900191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eec989-72e8-4af7-bd0e-70a683fb740b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.903408 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1147/2290] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:41 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:41.905932 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc83d-ee2f-44a6-bcbf-46251e089541 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.911408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1144/2291] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:41.915886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eec989-72e8-4af7-bd0e-70a683fb740b admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.918525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfc83d-ee2f-44a6-bcbf-46251e089541 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.923596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de14d1-0291-4b73-a836-ee8c63f87bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.924206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de14d1-0291-4b73-a836-ee8c63f87bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.925609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de14d1-0291-4b73-a836-ee8c63f87bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.925609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de14d1-0291-4b73-a836-ee8c63f87bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.926177 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de14d1-0291-4b73-a836-ee8c63f87bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.927749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eec989-72e8-4af7-bd0e-70a683fb740b admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.927884 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de14d1-0291-4b73-a836-ee8c63f87bd0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.933161 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b286a-db82-4fbf-b9d9-30e58851b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.933872 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b286a-db82-4fbf-b9d9-30e58851b585 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.934160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b286a-db82-4fbf-b9d9-30e58851b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.934477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45b286a-db82-4fbf-b9d9-30e58851b585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.935153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b286a-db82-4fbf-b9d9-30e58851b585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.936850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45b286a-db82-4fbf-b9d9-30e58851b585 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.943360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1148/2292] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:41.944681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de14d1-0291-4b73-a836-ee8c63f87bd0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.950572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1145/2293] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:41.955102 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b286a-db82-4fbf-b9d9-30e58851b585 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.956538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de14d1-0291-4b73-a836-ee8c63f87bd0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.961198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b92d1-6e4d-4fd4-87ac-b6db896d917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.961798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b92d1-6e4d-4fd4-87ac-b6db896d917d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.962084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b92d1-6e4d-4fd4-87ac-b6db896d917d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.962387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b92d1-6e4d-4fd4-87ac-b6db896d917d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.963033 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b92d1-6e4d-4fd4-87ac-b6db896d917d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:41.964657 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b92d1-6e4d-4fd4-87ac-b6db896d917d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:41.966248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45b286a-db82-4fbf-b9d9-30e58851b585 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.970922 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b3cfa-abfc-4fd6-bcb5-8f0eef7e0556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:41.972441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3cfa-abfc-4fd6-bcb5-8f0eef7e0556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:41.973345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3cfa-abfc-4fd6-bcb5-8f0eef7e0556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:41.973606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b3cfa-abfc-4fd6-bcb5-8f0eef7e0556 admin admin] PATH_INFO: `/v3/projects/3298bfce43384776baf7fab0dbf64fc7/users/7b9c138e52a44bc88a07bd2d3f2b53f3/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:41.976194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1149/2294] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:41.982297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b92d1-6e4d-4fd4-87ac-b6db896d917d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:41.993301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b92d1-6e4d-4fd4-87ac-b6db896d917d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:41.999311 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b3cfa-abfc-4fd6-bcb5-8f0eef7e0556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3298bfce43384776baf7fab0dbf64fc7, user_id=7b9c138e52a44bc88a07bd2d3f2b53f3, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.002702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b3cfa-abfc-4fd6-bcb5-8f0eef7e0556 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.003716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189e371-2228-45f8-a0af-5603c53bf54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.004396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e371-2228-45f8-a0af-5603c53bf54e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.004681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e371-2228-45f8-a0af-5603c53bf54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.005067 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189e371-2228-45f8-a0af-5603c53bf54e admin admin] PATH_INFO: `/v3/projects/49b426c946444937b56adfc4af5ea223/users/b7a68533d00f4ee4afe653569a6506b9/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.006334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:42.020874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1146/2295] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:41 2022] PUT /identity/v3/projects/3298bfce43384776baf7fab0dbf64fc7/users/7b9c138e52a44bc88a07bd2d3f2b53f3/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:42.024133 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189e371-2228-45f8-a0af-5603c53bf54e admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b426c946444937b56adfc4af5ea223, user_id=b7a68533d00f4ee4afe653569a6506b9, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.026876 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189e371-2228-45f8-a0af-5603c53bf54e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.031784 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2bcbc-c5bd-4835-8cb4-3c9539e80c8c None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.032156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2bcbc-c5bd-4835-8cb4-3c9539e80c8c None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.032438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2bcbc-c5bd-4835-8cb4-3c9539e80c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.043344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1150/2296] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:41 2022] PUT /identity/v3/projects/49b426c946444937b56adfc4af5ea223/users/b7a68533d00f4ee4afe653569a6506b9/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:42.054950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f6015-8e46-4edd-9880-007a2accdf10 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.055267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f6015-8e46-4edd-9880-007a2accdf10 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.055571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f6015-8e46-4edd-9880-007a2accdf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.070380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-fde2bcbc-c5bd-4835-8cb4-3c9539e80c8c None None] MFA Rules not processed for user `7b9c138e52a44bc88a07bd2d3f2b53f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:42.075796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fde2bcbc-c5bd-4835-8cb4-3c9539e80c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:42.090413 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-209f6015-8e46-4edd-9880-007a2accdf10 None None] MFA Rules not processed for user `b7a68533d00f4ee4afe653569a6506b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:42.092130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-209f6015-8e46-4edd-9880-007a2accdf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:42.147167 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1147/2297] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:42.151393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2bcbc-c5bd-4835-8cb4-3c9539e80c8c None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.164441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1151/2298] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:42.176518 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f6015-8e46-4edd-9880-007a2accdf10 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.178250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-209f6015-8e46-4edd-9880-007a2accdf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:42.211522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:42.224941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2bcbc-c5bd-4835-8cb4-3c9539e80c8c None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.229569 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5510c95-c2f9-4b33-a8bd-f311e1932ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.230180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5510c95-c2f9-4b33-a8bd-f311e1932ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.230454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5510c95-c2f9-4b33-a8bd-f311e1932ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.230726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5510c95-c2f9-4b33-a8bd-f311e1932ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.231413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5510c95-c2f9-4b33-a8bd-f311e1932ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.233387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5510c95-c2f9-4b33-a8bd-f311e1932ef7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.255917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f6015-8e46-4edd-9880-007a2accdf10 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.261476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fe43d-9447-4119-aac5-ade53f0caf37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.262220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe43d-9447-4119-aac5-ade53f0caf37 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.262566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe43d-9447-4119-aac5-ade53f0caf37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.263008 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fe43d-9447-4119-aac5-ade53f0caf37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.267091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1148/2299] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:42 2022] POST /identity/v3/projects => generated 357 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:19:42.270824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5510c95-c2f9-4b33-a8bd-f311e1932ef7 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.273788 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fe43d-9447-4119-aac5-ade53f0caf37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.275364 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fe43d-9447-4119-aac5-ade53f0caf37 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.286160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1152/2300] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:19:42.342202 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5510c95-c2f9-4b33-a8bd-f311e1932ef7 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.347608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e77da2-fcc3-407b-b273-116c4b23993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.348218 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e77da2-fcc3-407b-b273-116c4b23993a admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.348497 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e77da2-fcc3-407b-b273-116c4b23993a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.348770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e77da2-fcc3-407b-b273-116c4b23993a admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.349404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e77da2-fcc3-407b-b273-116c4b23993a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.351374 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e77da2-fcc3-407b-b273-116c4b23993a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.368596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:42.378820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1149/2301] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:42 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:42.379250 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fe43d-9447-4119-aac5-ade53f0caf37 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.391720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fe43d-9447-4119-aac5-ade53f0caf37 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.397084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d47f7-def0-4308-ab92-d2b0825c8104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.397707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d47f7-def0-4308-ab92-d2b0825c8104 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.397986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d47f7-def0-4308-ab92-d2b0825c8104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.398261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d47f7-def0-4308-ab92-d2b0825c8104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.398903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d47f7-def0-4308-ab92-d2b0825c8104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.400597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d47f7-def0-4308-ab92-d2b0825c8104 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.410869 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1153/2302] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:19:42.415573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e77da2-fcc3-407b-b273-116c4b23993a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.425753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e77da2-fcc3-407b-b273-116c4b23993a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.430415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832ba9e-7d07-4fcb-92fe-399fd63d3758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.431109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832ba9e-7d07-4fcb-92fe-399fd63d3758 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.431390 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832ba9e-7d07-4fcb-92fe-399fd63d3758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.431671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832ba9e-7d07-4fcb-92fe-399fd63d3758 admin admin] PATH_INFO: `/v3/projects/5454d2b816de4fe8b4cc759915f830d4/users/4edc78e892ce4d80a3b0c816c7b0c366/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.448455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832ba9e-7d07-4fcb-92fe-399fd63d3758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5454d2b816de4fe8b4cc759915f830d4, user_id=4edc78e892ce4d80a3b0c816c7b0c366, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.451622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832ba9e-7d07-4fcb-92fe-399fd63d3758 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.473046 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1150/2303] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:42 2022] PUT /identity/v3/projects/5454d2b816de4fe8b4cc759915f830d4/users/4edc78e892ce4d80a3b0c816c7b0c366/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:19:42.477114 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d47f7-def0-4308-ab92-d2b0825c8104 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.489378 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d47f7-def0-4308-ab92-d2b0825c8104 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.551810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aec95-25e4-40f8-957f-56512d7c191d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.552764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aec95-25e4-40f8-957f-56512d7c191d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.552764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aec95-25e4-40f8-957f-56512d7c191d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.553180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aec95-25e4-40f8-957f-56512d7c191d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.553746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aec95-25e4-40f8-957f-56512d7c191d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.555567 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aec95-25e4-40f8-957f-56512d7c191d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.568380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1154/2304] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:42.572263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832ba9e-7d07-4fcb-92fe-399fd63d3758 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.585157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832ba9e-7d07-4fcb-92fe-399fd63d3758 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.590663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861f420-d3c9-4cfb-9316-9009f99da295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.591430 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861f420-d3c9-4cfb-9316-9009f99da295 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.591895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861f420-d3c9-4cfb-9316-9009f99da295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.592259 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9861f420-d3c9-4cfb-9316-9009f99da295 admin admin] PATH_INFO: `/v3/projects/5454d2b816de4fe8b4cc759915f830d4/users/4edc78e892ce4d80a3b0c816c7b0c366/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.594423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861f420-d3c9-4cfb-9316-9009f99da295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5454d2b816de4fe8b4cc759915f830d4, user_id=4edc78e892ce4d80a3b0c816c7b0c366, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.598131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9861f420-d3c9-4cfb-9316-9009f99da295 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.618356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1151/2305] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:42 2022] PUT /identity/v3/projects/5454d2b816de4fe8b4cc759915f830d4/users/4edc78e892ce4d80a3b0c816c7b0c366/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:42.630028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f30be2-f2f2-415e-827b-88403dd116e9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.630319 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f30be2-f2f2-415e-827b-88403dd116e9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.630598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f30be2-f2f2-415e-827b-88403dd116e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.659558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-05f30be2-f2f2-415e-827b-88403dd116e9 None None] MFA Rules not processed for user `4edc78e892ce4d80a3b0c816c7b0c366`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:42.660922 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-05f30be2-f2f2-415e-827b-88403dd116e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:42.730396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1155/2306] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:42.742219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861f420-d3c9-4cfb-9316-9009f99da295 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.798909 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9861f420-d3c9-4cfb-9316-9009f99da295 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.803015 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.803636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.803918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.804193 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.814222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:42.815863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:42.825490 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1152/2307] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:42.913888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f30be2-f2f2-415e-827b-88403dd116e9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:42.923823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f30be2-f2f2-415e-827b-88403dd116e9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:42.927985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:42.928597 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:42.928955 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:42.929143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:42.967351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:43.000708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:43.002164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:43.013873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1156/2308] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:44.230860 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:44.232435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:44.297691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e5415-0788-46c5-8dba-dbe53e071894 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:44.302282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:44.302925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:44.303243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:44.303521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:44.305281 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:44.384282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:44.385919 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:44.396548 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1153/2309] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:48.533943 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:48.536358 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:48.551681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ac54c-85ce-46d2-a51b-b6c25a87d4f0 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:48.556572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:48.557208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:48.557488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:48.557791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] PATH_INFO: `/v3/users/3e02f6f701324af6b53762569f1a9d21` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:48.558925 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e02f6f701324af6b53762569f1a9d21)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:48.560945 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:48.612592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e02f6f701324af6b53762569f1a9d21'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:48.626851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e02f6f701324af6b53762569f1a9d21'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:48.628640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e02f6f701324af6b53762569f1a9d21'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:48.638253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e02f6f701324af6b53762569f1a9d21'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:48.638878 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d2aef535-c0f7-4e87-9bea-5a57e9a20633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e02f6f701324af6b53762569f1a9d21'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:48.655284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1157/2310] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:48 2022] DELETE /identity/v3/users/3e02f6f701324af6b53762569f1a9d21 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:48.846506 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:48.873004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62eddde-328e-4294-af42-f540c70c03a4 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:48.890162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:48.943367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:48.944055 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:48.944329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:48.944605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] PATH_INFO: `/v3/projects/b736b09b6ee2492590cbc23db48752f4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:48.945778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b736b09b6ee2492590cbc23db48752f4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:48.947921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:49.005950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b736b09b6ee2492590cbc23db48752f4', 'request_id': 'req-4188476e-c33e-47a2-b7ac-9f79ac6a0850'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:49.013694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b736b09b6ee2492590cbc23db48752f4', 'request_id': 'req-4188476e-c33e-47a2-b7ac-9f79ac6a0850'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:49.024798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b736b09b6ee2492590cbc23db48752f4', 'request_id': 'req-4188476e-c33e-47a2-b7ac-9f79ac6a0850'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:49.036437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] The token cache is being invalidate because project b736b09b6ee2492590cbc23db48752f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:19:49.036731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:49.037392 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4188476e-c33e-47a2-b7ac-9f79ac6a0850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:49.038897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1154/2311] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:19:48 2022] DELETE /identity/v3/projects/b736b09b6ee2492590cbc23db48752f4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:49.043933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c7d9-7e49-4be6-bcaf-c986c3ff3b4f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.044219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c7d9-7e49-4be6-bcaf-c986c3ff3b4f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.044496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c7d9-7e49-4be6-bcaf-c986c3ff3b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.077247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d146c7d9-7e49-4be6-bcaf-c986c3ff3b4f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:49.078467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d146c7d9-7e49-4be6-bcaf-c986c3ff3b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:49.143106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1158/2312] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:19:49.150846 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fddb9-4012-4fe3-a95a-2812b87f8c1b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.151171 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fddb9-4012-4fe3-a95a-2812b87f8c1b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.151453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fddb9-4012-4fe3-a95a-2812b87f8c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.195795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7b8fddb9-4012-4fe3-a95a-2812b87f8c1b None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:49.197073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7b8fddb9-4012-4fe3-a95a-2812b87f8c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:49.208705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1155/2313] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:49.212782 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146c7d9-7e49-4be6-bcaf-c986c3ff3b4f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:49.224772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146c7d9-7e49-4be6-bcaf-c986c3ff3b4f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:49.232429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:49.233086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.233374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.233655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.234785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:19:49.234785 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:49.236709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:49.251386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1159/2314] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:19:49.256189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fddb9-4012-4fe3-a95a-2812b87f8c1b None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:49.268954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fddb9-4012-4fe3-a95a-2812b87f8c1b None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:49.275407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a036ba-3da3-4d21-9c91-20901f872dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:49.275975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a036ba-3da3-4d21-9c91-20901f872dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.276433 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a036ba-3da3-4d21-9c91-20901f872dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.276723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a036ba-3da3-4d21-9c91-20901f872dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.277384 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a036ba-3da3-4d21-9c91-20901f872dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:49.279770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a036ba-3da3-4d21-9c91-20901f872dc8 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:49.317628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1156/2315] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:19:49 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:49.318907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:49.329367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c9dc9-7c87-429e-b978-1dd20ab08707 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:49.376333 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:49.420972 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bf57a-c864-404b-ae32-a64bd6a5a1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:49.422255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bf57a-c864-404b-ae32-a64bd6a5a1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.422540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bf57a-c864-404b-ae32-a64bd6a5a1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.422817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bf57a-c864-404b-ae32-a64bd6a5a1d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.423496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bf57a-c864-404b-ae32-a64bd6a5a1d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:49.427484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1bf57a-c864-404b-ae32-a64bd6a5a1d7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:49.451984 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1160/2316] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:19:49 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:19:49.457457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a036ba-3da3-4d21-9c91-20901f872dc8 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:49.476827 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a036ba-3da3-4d21-9c91-20901f872dc8 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:49.483834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee699b5a-e5c8-46e6-8f24-409684ea0f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:49.484035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee699b5a-e5c8-46e6-8f24-409684ea0f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.484182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee699b5a-e5c8-46e6-8f24-409684ea0f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.484468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee699b5a-e5c8-46e6-8f24-409684ea0f4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.485124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee699b5a-e5c8-46e6-8f24-409684ea0f4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:49.487001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee699b5a-e5c8-46e6-8f24-409684ea0f4f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:49.499804 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1157/2317] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:49.503904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bf57a-c864-404b-ae32-a64bd6a5a1d7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:49.536416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bf57a-c864-404b-ae32-a64bd6a5a1d7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:49.548919 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefead53-cd37-4589-a822-c3a788d8113a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:49.550308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefead53-cd37-4589-a822-c3a788d8113a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.550962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefead53-cd37-4589-a822-c3a788d8113a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.551403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefead53-cd37-4589-a822-c3a788d8113a admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.552564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefead53-cd37-4589-a822-c3a788d8113a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:49.554505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefead53-cd37-4589-a822-c3a788d8113a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:49.570148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1161/2318] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:19:49.572890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee699b5a-e5c8-46e6-8f24-409684ea0f4f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:49.585692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee699b5a-e5c8-46e6-8f24-409684ea0f4f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:49.590262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:49.591575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.592031 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.593355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] PATH_INFO: `/v3/projects/4edaa125c0154653b7b698cc42630b0d/users/abfbfe6583274bd88e90d7007a7c90a8/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.612230 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4edaa125c0154653b7b698cc42630b0d, user_id=abfbfe6583274bd88e90d7007a7c90a8, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:49.617028 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:49.635952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1158/2319] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:19:49 2022] PUT /identity/v3/projects/4edaa125c0154653b7b698cc42630b0d/users/abfbfe6583274bd88e90d7007a7c90a8/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:49.659449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2600b3-bcdf-4c8f-b22b-17b60bf244f2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.660051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2600b3-bcdf-4c8f-b22b-17b60bf244f2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.660488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2600b3-bcdf-4c8f-b22b-17b60bf244f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.690954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4c2600b3-bcdf-4c8f-b22b-17b60bf244f2 None None] MFA Rules not processed for user `abfbfe6583274bd88e90d7007a7c90a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:49.692361 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4c2600b3-bcdf-4c8f-b22b-17b60bf244f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:49.768866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1162/2320] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:49.779930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:49.781763 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:49.849247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:49.929025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9aa4c-d979-42d1-b963-278e0473e330 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:49.940576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625af00d-21ae-4c33-a33c-be40d84dac76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:49.941217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625af00d-21ae-4c33-a33c-be40d84dac76 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:49.941499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625af00d-21ae-4c33-a33c-be40d84dac76 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:49.941778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625af00d-21ae-4c33-a33c-be40d84dac76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:49.955177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625af00d-21ae-4c33-a33c-be40d84dac76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:49.958629 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625af00d-21ae-4c33-a33c-be40d84dac76 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:49.972298 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1159/2321] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:19:59.626480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2600b3-bcdf-4c8f-b22b-17b60bf244f2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:59.688178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2600b3-bcdf-4c8f-b22b-17b60bf244f2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:59.692930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:59.693564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:59.693851 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:59.694132 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] PATH_INFO: `/v3/users/abfbfe6583274bd88e90d7007a7c90a8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:59.695293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfbfe6583274bd88e90d7007a7c90a8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:19:59.697233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:19:59.751630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:19:59.755682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfbfe6583274bd88e90d7007a7c90a8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:59.769222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfbfe6583274bd88e90d7007a7c90a8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:59.769862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfbfe6583274bd88e90d7007a7c90a8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:59.777619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfbfe6583274bd88e90d7007a7c90a8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:59.778280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfbfe6583274bd88e90d7007a7c90a8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:19:59.792199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1163/2322] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:19:59 2022] DELETE /identity/v3/users/abfbfe6583274bd88e90d7007a7c90a8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:19:59.796440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be53f9-be0c-46e4-a47a-b01f26d5c0ba None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:59.796930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be53f9-be0c-46e4-a47a-b01f26d5c0ba None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:59.797038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be53f9-be0c-46e4-a47a-b01f26d5c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:19:59.830673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a4be53f9-be0c-46e4-a47a-b01f26d5c0ba None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:19:59.832192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a4be53f9-be0c-46e4-a47a-b01f26d5c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:59.896431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1160/2323] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:19:59.906598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:19:59.908269 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:19:59.984692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55670186-eb06-4800-9c51-a8ca40153e4f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:19:59.988643 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361fb7e-5f60-49fe-a6ba-ce666c537762 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:19:59.989322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361fb7e-5f60-49fe-a6ba-ce666c537762 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:19:59.989678 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361fb7e-5f60-49fe-a6ba-ce666c537762 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:19:59.990027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361fb7e-5f60-49fe-a6ba-ce666c537762 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.003896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361fb7e-5f60-49fe-a6ba-ce666c537762 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:00.005179 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361fb7e-5f60-49fe-a6ba-ce666c537762 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:00.019190 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1164/2324] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:19:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:00.362012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be53f9-be0c-46e4-a47a-b01f26d5c0ba None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:00.364457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a4be53f9-be0c-46e4-a47a-b01f26d5c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:00.379719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be53f9-be0c-46e4-a47a-b01f26d5c0ba None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:00.385418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:00.386474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.386828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.387359 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] PATH_INFO: `/v3/projects/4edaa125c0154653b7b698cc42630b0d` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.388679 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4edaa125c0154653b7b698cc42630b0d)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:00.391196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:00.468177 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4edaa125c0154653b7b698cc42630b0d', 'request_id': 'req-70d819e7-fc7e-4d98-b0f1-f113635209a1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:00.476060 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4edaa125c0154653b7b698cc42630b0d', 'request_id': 'req-70d819e7-fc7e-4d98-b0f1-f113635209a1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:00.485926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4edaa125c0154653b7b698cc42630b0d', 'request_id': 'req-70d819e7-fc7e-4d98-b0f1-f113635209a1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:00.501051 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] The token cache is being invalidate because project 4edaa125c0154653b7b698cc42630b0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:00.501652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:00.502623 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-70d819e7-fc7e-4d98-b0f1-f113635209a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:00.505566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1161/2325] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:00 2022] DELETE /identity/v3/projects/4edaa125c0154653b7b698cc42630b0d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:00.513577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39c8b-3f12-46bc-a080-1b7d8a3daf05 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.513967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39c8b-3f12-46bc-a080-1b7d8a3daf05 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.514299 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b39c8b-3f12-46bc-a080-1b7d8a3daf05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.535630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-85b39c8b-3f12-46bc-a080-1b7d8a3daf05 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:00.536861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-85b39c8b-3f12-46bc-a080-1b7d8a3daf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:00.597747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1165/2326] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:00.605628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda73381-e7fd-489f-bfb7-4583b591b8e2 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.605862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda73381-e7fd-489f-bfb7-4583b591b8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.606196 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda73381-e7fd-489f-bfb7-4583b591b8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.627672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-fda73381-e7fd-489f-bfb7-4583b591b8e2 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:00.630156 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fda73381-e7fd-489f-bfb7-4583b591b8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:00.645701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1162/2327] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:00.647373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b39c8b-3f12-46bc-a080-1b7d8a3daf05 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:00.658575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b39c8b-3f12-46bc-a080-1b7d8a3daf05 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:00.664303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:00.665359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.665359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.665569 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.666275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:00.666581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:00.668336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:00.684211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1166/2328] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:20:00.689740 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda73381-e7fd-489f-bfb7-4583b591b8e2 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:00.701163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda73381-e7fd-489f-bfb7-4583b591b8e2 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:00.708019 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cb61b-e41c-469d-86ad-1bf90e4a5dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:00.709393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cb61b-e41c-469d-86ad-1bf90e4a5dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.709692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cb61b-e41c-469d-86ad-1bf90e4a5dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.710124 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93cb61b-e41c-469d-86ad-1bf90e4a5dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.710926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cb61b-e41c-469d-86ad-1bf90e4a5dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:00.713088 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93cb61b-e41c-469d-86ad-1bf90e4a5dc6 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:00.748669 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1163/2329] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:00 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:00.752790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:00.765024 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84daa5dd-f86c-4f62-a347-d507bec7b82e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:00.827981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:00.844426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6762b8-3618-4abe-9f74-57abea2eb767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:00.845092 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6762b8-3618-4abe-9f74-57abea2eb767 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.845380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6762b8-3618-4abe-9f74-57abea2eb767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.845656 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6762b8-3618-4abe-9f74-57abea2eb767 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.846317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6762b8-3618-4abe-9f74-57abea2eb767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:00.849414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6762b8-3618-4abe-9f74-57abea2eb767 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:00.882730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1167/2330] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:00 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:20:00.887122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cb61b-e41c-469d-86ad-1bf90e4a5dc6 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:00.900377 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93cb61b-e41c-469d-86ad-1bf90e4a5dc6 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:00.910408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a4279-e70f-4b45-86d6-f2f86eebab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:00.911975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a4279-e70f-4b45-86d6-f2f86eebab3a admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.912508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a4279-e70f-4b45-86d6-f2f86eebab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.916017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a4279-e70f-4b45-86d6-f2f86eebab3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.917299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a4279-e70f-4b45-86d6-f2f86eebab3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:00.919231 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a4279-e70f-4b45-86d6-f2f86eebab3a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:00.934380 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1164/2331] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:20:00.938430 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6762b8-3618-4abe-9f74-57abea2eb767 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:00.949672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6762b8-3618-4abe-9f74-57abea2eb767 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:00.954873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2fc589-adc2-4c7f-85b1-72e1706ef5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:00.955540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc589-adc2-4c7f-85b1-72e1706ef5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.955836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc589-adc2-4c7f-85b1-72e1706ef5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.956705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2fc589-adc2-4c7f-85b1-72e1706ef5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:00.957572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fc589-adc2-4c7f-85b1-72e1706ef5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:00.959300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2fc589-adc2-4c7f-85b1-72e1706ef5e4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:00.972410 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1168/2332] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:20:00.977685 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a4279-e70f-4b45-86d6-f2f86eebab3a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:00.989722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a4279-e70f-4b45-86d6-f2f86eebab3a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:00.994850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:00.995600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:00.995912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:00.996201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] PATH_INFO: `/v3/projects/5a26e1adc98e44a6b744493552d8319f/users/a7d72a421d8f49d1bfaa7946c961eefa/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:01.007986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:01.019541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a26e1adc98e44a6b744493552d8319f, user_id=a7d72a421d8f49d1bfaa7946c961eefa, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:01.022707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:01.048683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1165/2333] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:00 2022] PUT /identity/v3/projects/5a26e1adc98e44a6b744493552d8319f/users/a7d72a421d8f49d1bfaa7946c961eefa/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:01.061235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c12fc-a8c8-4138-9e5d-b37b58ce3596 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:01.061533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c12fc-a8c8-4138-9e5d-b37b58ce3596 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:01.061817 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c12fc-a8c8-4138-9e5d-b37b58ce3596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:01.091217 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-340c12fc-a8c8-4138-9e5d-b37b58ce3596 None None] MFA Rules not processed for user `a7d72a421d8f49d1bfaa7946c961eefa`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:01.092435 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-340c12fc-a8c8-4138-9e5d-b37b58ce3596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:01.161707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1169/2334] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:01.172217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:01.173956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:01.256431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ce692-a593-4c19-8941-6a5abfbd8b99 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:01.262284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350f03f-f987-4211-9d55-f24725d72fd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:01.262901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350f03f-f987-4211-9d55-f24725d72fd8 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:01.263222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350f03f-f987-4211-9d55-f24725d72fd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:01.263504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350f03f-f987-4211-9d55-f24725d72fd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:01.271573 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350f03f-f987-4211-9d55-f24725d72fd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:01.273188 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350f03f-f987-4211-9d55-f24725d72fd8 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:01.282081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1166/2335] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:01.678295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c12fc-a8c8-4138-9e5d-b37b58ce3596 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:01.679802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-340c12fc-a8c8-4138-9e5d-b37b58ce3596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:01.767167 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c12fc-a8c8-4138-9e5d-b37b58ce3596 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:01.771896 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:01.772534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:01.772818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:01.773101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] PATH_INFO: `/v3/users/7b9c138e52a44bc88a07bd2d3f2b53f3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:01.774229 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9c138e52a44bc88a07bd2d3f2b53f3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:01.776202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:01.824549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9c138e52a44bc88a07bd2d3f2b53f3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:01.834730 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:01.837226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9c138e52a44bc88a07bd2d3f2b53f3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:01.838041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9c138e52a44bc88a07bd2d3f2b53f3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:01.849531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9c138e52a44bc88a07bd2d3f2b53f3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:01.850158 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9c138e52a44bc88a07bd2d3f2b53f3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:01.863766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1170/2336] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:01 2022] DELETE /identity/v3/users/7b9c138e52a44bc88a07bd2d3f2b53f3 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:01.868230 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1d732-98ac-4842-a9a1-962d6ce916eb None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:01.868522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1d732-98ac-4842-a9a1-962d6ce916eb None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:01.868800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1d732-98ac-4842-a9a1-962d6ce916eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:01.890403 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-adb1d732-98ac-4842-a9a1-962d6ce916eb None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:01.891928 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-adb1d732-98ac-4842-a9a1-962d6ce916eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:01.954130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1167/2337] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:01.964262 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:01.965661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:02.024775 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c6f00-060d-48e0-9986-f9f0e7c25609 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:02.028647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381aad3-9c07-4f74-b88f-46c90aefd13b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:02.029268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381aad3-9c07-4f74-b88f-46c90aefd13b service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:02.029551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381aad3-9c07-4f74-b88f-46c90aefd13b service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:02.029843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381aad3-9c07-4f74-b88f-46c90aefd13b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:02.040655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381aad3-9c07-4f74-b88f-46c90aefd13b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:02.042088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381aad3-9c07-4f74-b88f-46c90aefd13b service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:02.051740 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1171/2338] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:20:02.400095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1d732-98ac-4842-a9a1-962d6ce916eb None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:02.401744 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-adb1d732-98ac-4842-a9a1-962d6ce916eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:02.421387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1d732-98ac-4842-a9a1-962d6ce916eb None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:02.426681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:02.427444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:02.427620 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:02.427901 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] PATH_INFO: `/v3/projects/3298bfce43384776baf7fab0dbf64fc7` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:02.429427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3298bfce43384776baf7fab0dbf64fc7)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:02.431502 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:02.483064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3298bfce43384776baf7fab0dbf64fc7', 'request_id': 'req-f94ee5b6-a7e6-47e0-9aa2-676300145b07'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.489442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3298bfce43384776baf7fab0dbf64fc7', 'request_id': 'req-f94ee5b6-a7e6-47e0-9aa2-676300145b07'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.497285 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3298bfce43384776baf7fab0dbf64fc7', 'request_id': 'req-f94ee5b6-a7e6-47e0-9aa2-676300145b07'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.502072 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:02.510063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] The token cache is being invalidate because project 3298bfce43384776baf7fab0dbf64fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:02.510370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.511507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.513145 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1168/2339] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:02 2022] DELETE /identity/v3/projects/3298bfce43384776baf7fab0dbf64fc7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:02.517178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381aad3-9c07-4f74-b88f-46c90aefd13b service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:02.527651 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381aad3-9c07-4f74-b88f-46c90aefd13b service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:02.587564 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:02.588201 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:02.588481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:02.588778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] PATH_INFO: `/v3/users/4edc78e892ce4d80a3b0c816c7b0c366` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:02.589898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4edc78e892ce4d80a3b0c816c7b0c366)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:02.591904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:02.660219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4edc78e892ce4d80a3b0c816c7b0c366'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.674692 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4edc78e892ce4d80a3b0c816c7b0c366'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.675629 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4edc78e892ce4d80a3b0c816c7b0c366'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.683044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4edc78e892ce4d80a3b0c816c7b0c366'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.683688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93ca139f-7e69-41e8-86d5-82562f028fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4edc78e892ce4d80a3b0c816c7b0c366'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:02.692636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:02.698353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1172/2340] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:02 2022] DELETE /identity/v3/users/4edc78e892ce4d80a3b0c816c7b0c366 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:03.044930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.046897 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:03.073090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee5b6-a7e6-47e0-9aa2-676300145b07 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.139243 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.140039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.140509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.140792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] PATH_INFO: `/v3/projects/5454d2b816de4fe8b4cc759915f830d4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.141940 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5454d2b816de4fe8b4cc759915f830d4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.144546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:03.194045 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5454d2b816de4fe8b4cc759915f830d4', 'request_id': 'req-5d20c585-4159-4515-b956-f7dadf6df566'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:03.203748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5454d2b816de4fe8b4cc759915f830d4', 'request_id': 'req-5d20c585-4159-4515-b956-f7dadf6df566'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:03.211722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5454d2b816de4fe8b4cc759915f830d4', 'request_id': 'req-5d20c585-4159-4515-b956-f7dadf6df566'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:03.222683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] The token cache is being invalidate because project 5454d2b816de4fe8b4cc759915f830d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:03.222981 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:03.223655 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-5d20c585-4159-4515-b956-f7dadf6df566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:03.225172 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1169/2341] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:03 2022] DELETE /identity/v3/projects/5454d2b816de4fe8b4cc759915f830d4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:03.230142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed545c-2155-44fa-83c6-d49f11b0151a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.230428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed545c-2155-44fa-83c6-d49f11b0151a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.230705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed545c-2155-44fa-83c6-d49f11b0151a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.250527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-29ed545c-2155-44fa-83c6-d49f11b0151a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:03.251779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-29ed545c-2155-44fa-83c6-d49f11b0151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:03.313104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1173/2342] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:03.321926 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5206a9b-044d-46b6-844b-17f891aefcc7 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.322220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5206a9b-044d-46b6-844b-17f891aefcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.322500 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5206a9b-044d-46b6-844b-17f891aefcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.343657 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f5206a9b-044d-46b6-844b-17f891aefcc7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:03.344857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f5206a9b-044d-46b6-844b-17f891aefcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:03.353758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1170/2343] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:03.358144 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed545c-2155-44fa-83c6-d49f11b0151a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.372080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed545c-2155-44fa-83c6-d49f11b0151a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.376992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.377615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.377918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.378183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.378853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:03.379393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.381039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:03.393671 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1174/2344] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:20:03.398745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5206a9b-044d-46b6-844b-17f891aefcc7 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.408312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5206a9b-044d-46b6-844b-17f891aefcc7 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.414155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051170f-8ef8-4c42-bd3f-ea0eaca9a005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.414774 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051170f-8ef8-4c42-bd3f-ea0eaca9a005 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.415087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051170f-8ef8-4c42-bd3f-ea0eaca9a005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.415352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051170f-8ef8-4c42-bd3f-ea0eaca9a005 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.416005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051170f-8ef8-4c42-bd3f-ea0eaca9a005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.417942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051170f-8ef8-4c42-bd3f-ea0eaca9a005 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:03.451598 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1171/2345] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:03 2022] POST /identity/v3/projects => generated 391 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:03.455893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.466714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fabca-ac46-4458-b801-26aa0de147d2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.529880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f78e2-c42a-4b6b-86b9-795691458e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.530504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f78e2-c42a-4b6b-86b9-795691458e72 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.530792 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f78e2-c42a-4b6b-86b9-795691458e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.531098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f78e2-c42a-4b6b-86b9-795691458e72 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.531914 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f78e2-c42a-4b6b-86b9-795691458e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.533920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f78e2-c42a-4b6b-86b9-795691458e72 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:03.556948 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1175/2346] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:03 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:03.560995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051170f-8ef8-4c42-bd3f-ea0eaca9a005 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.571227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051170f-8ef8-4c42-bd3f-ea0eaca9a005 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.575770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e1e7c-8eea-43d1-9272-8f40dba82e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.576358 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1e7c-8eea-43d1-9272-8f40dba82e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.576638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1e7c-8eea-43d1-9272-8f40dba82e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.576911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e1e7c-8eea-43d1-9272-8f40dba82e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.577542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e1e7c-8eea-43d1-9272-8f40dba82e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.579190 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e1e7c-8eea-43d1-9272-8f40dba82e5e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:03.589050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1172/2347] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:03.593484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f78e2-c42a-4b6b-86b9-795691458e72 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.606403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f78e2-c42a-4b6b-86b9-795691458e72 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.611434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d91cc65-dda8-41e9-a9f4-6d52337dbf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.612058 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91cc65-dda8-41e9-a9f4-6d52337dbf57 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.612949 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91cc65-dda8-41e9-a9f4-6d52337dbf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.613245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d91cc65-dda8-41e9-a9f4-6d52337dbf57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.613909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d91cc65-dda8-41e9-a9f4-6d52337dbf57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.615557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d91cc65-dda8-41e9-a9f4-6d52337dbf57 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:03.626392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1176/2348] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:03.630778 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e1e7c-8eea-43d1-9272-8f40dba82e5e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.641014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e1e7c-8eea-43d1-9272-8f40dba82e5e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.646290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c119fa0-c52b-47b2-9093-a6156eff37ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.646948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c119fa0-c52b-47b2-9093-a6156eff37ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.647261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c119fa0-c52b-47b2-9093-a6156eff37ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.647546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c119fa0-c52b-47b2-9093-a6156eff37ea admin admin] PATH_INFO: `/v3/projects/b636efc9fdd8414cb94268b52c1aea21/users/dab7d79197d04b5db9a381b9079a15ab/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.663997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c119fa0-c52b-47b2-9093-a6156eff37ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b636efc9fdd8414cb94268b52c1aea21, user_id=dab7d79197d04b5db9a381b9079a15ab, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.666756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c119fa0-c52b-47b2-9093-a6156eff37ea admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:03.689367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1173/2349] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:03 2022] PUT /identity/v3/projects/b636efc9fdd8414cb94268b52c1aea21/users/dab7d79197d04b5db9a381b9079a15ab/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:03.700461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8d8d6-b7fb-4939-88bf-5724d5ae5556 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.700935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8d8d6-b7fb-4939-88bf-5724d5ae5556 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.701064 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8d8d6-b7fb-4939-88bf-5724d5ae5556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.730223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d3a8d8d6-b7fb-4939-88bf-5724d5ae5556 None None] MFA Rules not processed for user `dab7d79197d04b5db9a381b9079a15ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:03.731476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d3a8d8d6-b7fb-4939-88bf-5724d5ae5556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:03.793707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:03.803859 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1177/2350] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:03.808957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c119fa0-c52b-47b2-9093-a6156eff37ea admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.836983 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:03.875918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c119fa0-c52b-47b2-9093-a6156eff37ea admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.880610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7b031-fa08-41bc-8b91-ad5061f7d2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.881219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7b031-fa08-41bc-8b91-ad5061f7d2cb admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.881503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7b031-fa08-41bc-8b91-ad5061f7d2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.881780 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7b031-fa08-41bc-8b91-ad5061f7d2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.882415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7b031-fa08-41bc-8b91-ad5061f7d2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.884360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7b031-fa08-41bc-8b91-ad5061f7d2cb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:03.911077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1174/2351] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:03 2022] POST /identity/v3/projects => generated 393 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:20:03.915194 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8d8d6-b7fb-4939-88bf-5724d5ae5556 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:03.926296 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8d8d6-b7fb-4939-88bf-5724d5ae5556 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:03.983904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849316-29a0-46e5-9658-08dcec1dcba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:03.984521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849316-29a0-46e5-9658-08dcec1dcba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:03.984809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849316-29a0-46e5-9658-08dcec1dcba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:03.985087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0849316-29a0-46e5-9658-08dcec1dcba8 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:03.985726 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0849316-29a0-46e5-9658-08dcec1dcba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:03.987855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0849316-29a0-46e5-9658-08dcec1dcba8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.011893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1178/2352] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:03 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:04.013013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7b031-fa08-41bc-8b91-ad5061f7d2cb admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.024182 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7b031-fa08-41bc-8b91-ad5061f7d2cb admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.028861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.029476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.029787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.030071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.030787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.032422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.044378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1175/2353] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:20:04.048517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849316-29a0-46e5-9658-08dcec1dcba8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.059040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0849316-29a0-46e5-9658-08dcec1dcba8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.063871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cab7ec-1a02-4a7f-b51b-901d48244643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.064534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cab7ec-1a02-4a7f-b51b-901d48244643 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.064819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cab7ec-1a02-4a7f-b51b-901d48244643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.065110 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cab7ec-1a02-4a7f-b51b-901d48244643 admin admin] PATH_INFO: `/v3/projects/a97e6de2128542088e5f2049fa7926c3/users/fbec283587774e7c9d73b83347e9b678/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.082585 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cab7ec-1a02-4a7f-b51b-901d48244643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a97e6de2128542088e5f2049fa7926c3, user_id=fbec283587774e7c9d73b83347e9b678, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.085339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cab7ec-1a02-4a7f-b51b-901d48244643 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.107163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.109371 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.110833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1179/2354] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:04 2022] PUT /identity/v3/projects/a97e6de2128542088e5f2049fa7926c3/users/fbec283587774e7c9d73b83347e9b678/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:04.115187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cab7ec-1a02-4a7f-b51b-901d48244643 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.122870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ce944-3db5-4973-beff-80446b87a505 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.180088 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cab7ec-1a02-4a7f-b51b-901d48244643 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.191081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.191081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.191081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.191081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] PATH_INFO: `/v3/users/a7d72a421d8f49d1bfaa7946c961eefa` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.191626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975a82-f421-45ca-b4ad-be693817d223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.191738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d72a421d8f49d1bfaa7946c961eefa)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.191841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975a82-f421-45ca-b4ad-be693817d223 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.191841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975a82-f421-45ca-b4ad-be693817d223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.191841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26975a82-f421-45ca-b4ad-be693817d223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.191841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26975a82-f421-45ca-b4ad-be693817d223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.192262 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.192452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26975a82-f421-45ca-b4ad-be693817d223 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.201753 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1180/2355] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:20:04.206672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975a82-f421-45ca-b4ad-be693817d223 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.217241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26975a82-f421-45ca-b4ad-be693817d223 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.222637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc947e5-b7cb-4740-9d6f-c4d313057efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.223488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc947e5-b7cb-4740-9d6f-c4d313057efe admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.223838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc947e5-b7cb-4740-9d6f-c4d313057efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.224145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc947e5-b7cb-4740-9d6f-c4d313057efe admin admin] PATH_INFO: `/v3/projects/a97e6de2128542088e5f2049fa7926c3/users/fbec283587774e7c9d73b83347e9b678/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.226418 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc947e5-b7cb-4740-9d6f-c4d313057efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a97e6de2128542088e5f2049fa7926c3, user_id=fbec283587774e7c9d73b83347e9b678, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.229245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc947e5-b7cb-4740-9d6f-c4d313057efe admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.239873 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d72a421d8f49d1bfaa7946c961eefa'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.240105 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1181/2356] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:04 2022] PUT /identity/v3/projects/a97e6de2128542088e5f2049fa7926c3/users/fbec283587774e7c9d73b83347e9b678/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:04.251581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d578b-9abb-436b-b099-bdbcbfa164a4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.251947 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d578b-9abb-436b-b099-bdbcbfa164a4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.252235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d578b-9abb-436b-b099-bdbcbfa164a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.255636 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d72a421d8f49d1bfaa7946c961eefa'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.256320 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d72a421d8f49d1bfaa7946c961eefa'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.264857 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d72a421d8f49d1bfaa7946c961eefa'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.281499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9468fcfd-b991-43b1-81bf-a080dc819264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d72a421d8f49d1bfaa7946c961eefa'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.287665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-317d578b-9abb-436b-b099-bdbcbfa164a4 None None] MFA Rules not processed for user `fbec283587774e7c9d73b83347e9b678`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:04.287957 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1176/2357] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:04 2022] DELETE /identity/v3/users/a7d72a421d8f49d1bfaa7946c961eefa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:04.288905 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-317d578b-9abb-436b-b099-bdbcbfa164a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.292165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee13ee-d07e-4802-94c6-7161d11f8804 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.292455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee13ee-d07e-4802-94c6-7161d11f8804 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.292732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee13ee-d07e-4802-94c6-7161d11f8804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.312454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-14ee13ee-d07e-4802-94c6-7161d11f8804 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:04.313666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-14ee13ee-d07e-4802-94c6-7161d11f8804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.322802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:04.324581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:04.368320 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1182/2358] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:04 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:04.381300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d578b-9abb-436b-b099-bdbcbfa164a4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.382698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-317d578b-9abb-436b-b099-bdbcbfa164a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.387302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1177/2359] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:04.398738 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee13ee-d07e-4802-94c6-7161d11f8804 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.400191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-14ee13ee-d07e-4802-94c6-7161d11f8804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.454881 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d578b-9abb-436b-b099-bdbcbfa164a4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.459235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.459845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.460149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.460447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.462680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee13ee-d07e-4802-94c6-7161d11f8804 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.466463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.467099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.467394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.467669 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.470839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.472951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.478931 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.480487 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.483210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1183/2360] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:04 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:20:04.491158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1178/2361] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:20:04.548482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.552113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.569375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8ed7b-c0ad-45c7-b5c2-6be79576ddd0 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.573437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3937a-f001-4115-8b22-232c929685bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.574068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3937a-f001-4115-8b22-232c929685bf service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.574350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3937a-f001-4115-8b22-232c929685bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.574612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3937a-f001-4115-8b22-232c929685bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.576456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6fc3937a-f001-4115-8b22-232c929685bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.647530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3937a-f001-4115-8b22-232c929685bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.648978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3937a-f001-4115-8b22-232c929685bf service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.658839 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1184/2362] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:04.856728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:04.858601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:04.883094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7554922-06fc-44fb-875c-dd279f0f925e service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:04.890335 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:04.891220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.891788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.892113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] PATH_INFO: `/v3/projects/5a26e1adc98e44a6b744493552d8319f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:04.894197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a26e1adc98e44a6b744493552d8319f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:04.897038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:04.949225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a26e1adc98e44a6b744493552d8319f', 'request_id': 'req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.956106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a26e1adc98e44a6b744493552d8319f', 'request_id': 'req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.964631 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a26e1adc98e44a6b744493552d8319f', 'request_id': 'req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.977794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] The token cache is being invalidate because project 5a26e1adc98e44a6b744493552d8319f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:04.977941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.978606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-8ed292a6-a47c-4eb2-96c3-4a1fea72d879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:04.980406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1179/2363] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:04 2022] DELETE /identity/v3/projects/5a26e1adc98e44a6b744493552d8319f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:04.985297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b778cb-0695-421c-aad1-293443c983df None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:04.985297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b778cb-0695-421c-aad1-293443c983df None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:04.985964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b778cb-0695-421c-aad1-293443c983df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.010292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f4b778cb-0695-421c-aad1-293443c983df None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:05.012684 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f4b778cb-0695-421c-aad1-293443c983df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:05.084383 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1185/2364] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:05.092917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818cafa-8c02-4d13-9ad5-b23ea6df0d29 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.093293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818cafa-8c02-4d13-9ad5-b23ea6df0d29 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.093695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818cafa-8c02-4d13-9ad5-b23ea6df0d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.114560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-6818cafa-8c02-4d13-9ad5-b23ea6df0d29 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:05.115807 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6818cafa-8c02-4d13-9ad5-b23ea6df0d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:05.123701 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1180/2365] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:05.128956 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b778cb-0695-421c-aad1-293443c983df None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.138951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b778cb-0695-421c-aad1-293443c983df None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.144134 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.144743 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.145028 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.145306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.145960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:05.146267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.147893 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.154514 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:05.160373 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1186/2366] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:20:05.164670 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818cafa-8c02-4d13-9ad5-b23ea6df0d29 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.173733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818cafa-8c02-4d13-9ad5-b23ea6df0d29 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.179431 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9882a-ec35-46ed-b4fa-50cf2ec1a224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.180026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9882a-ec35-46ed-b4fa-50cf2ec1a224 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.180303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9882a-ec35-46ed-b4fa-50cf2ec1a224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.180574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9882a-ec35-46ed-b4fa-50cf2ec1a224 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.181207 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9882a-ec35-46ed-b4fa-50cf2ec1a224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.183131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9882a-ec35-46ed-b4fa-50cf2ec1a224 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.211456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1181/2367] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:05 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:05.215809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.226433 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fcb0b-cccf-4649-b80e-e8034781fed3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.284862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7d796-afde-4845-9ce4-1997838105b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.285476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7d796-afde-4845-9ce4-1997838105b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.285761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7d796-afde-4845-9ce4-1997838105b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.286098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7d796-afde-4845-9ce4-1997838105b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.287291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7d796-afde-4845-9ce4-1997838105b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.289304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7d796-afde-4845-9ce4-1997838105b2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.310598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1187/2368] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:05 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:05.314961 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9882a-ec35-46ed-b4fa-50cf2ec1a224 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.325439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9882a-ec35-46ed-b4fa-50cf2ec1a224 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.330043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dde2a7-b3d1-4884-8eef-713fc3fc303b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.330646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde2a7-b3d1-4884-8eef-713fc3fc303b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.330930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde2a7-b3d1-4884-8eef-713fc3fc303b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.331821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dde2a7-b3d1-4884-8eef-713fc3fc303b admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.332466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dde2a7-b3d1-4884-8eef-713fc3fc303b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.334064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dde2a7-b3d1-4884-8eef-713fc3fc303b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.344090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1182/2369] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:05.348486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7d796-afde-4845-9ce4-1997838105b2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.358348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7d796-afde-4845-9ce4-1997838105b2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.363151 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb8764-4ce3-4e4b-8256-77545dd409eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.363752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb8764-4ce3-4e4b-8256-77545dd409eb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.364078 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb8764-4ce3-4e4b-8256-77545dd409eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.364360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb8764-4ce3-4e4b-8256-77545dd409eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.365026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb8764-4ce3-4e4b-8256-77545dd409eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.366681 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb8764-4ce3-4e4b-8256-77545dd409eb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.379838 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1188/2370] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:05.384008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dde2a7-b3d1-4884-8eef-713fc3fc303b admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.389414 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:05.396116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dde2a7-b3d1-4884-8eef-713fc3fc303b admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.401749 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbea71-d16d-4ec0-851f-865a58e63e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.402436 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbea71-d16d-4ec0-851f-865a58e63e5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.402721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbea71-d16d-4ec0-851f-865a58e63e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.402997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbea71-d16d-4ec0-851f-865a58e63e5c admin admin] PATH_INFO: `/v3/projects/1637b8c9d4364a578774bf9585f44ee2/users/3746c8dc35e046dcbbca4e7ffda46359/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.418520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbea71-d16d-4ec0-851f-865a58e63e5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1637b8c9d4364a578774bf9585f44ee2, user_id=3746c8dc35e046dcbbca4e7ffda46359, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.421248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbea71-d16d-4ec0-851f-865a58e63e5c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.434265 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1183/2371] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:05 2022] PUT /identity/v3/projects/1637b8c9d4364a578774bf9585f44ee2/users/3746c8dc35e046dcbbca4e7ffda46359/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:05.449139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58138fab-7878-4492-ac1d-f7ee46d375a8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.449427 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58138fab-7878-4492-ac1d-f7ee46d375a8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.449708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58138fab-7878-4492-ac1d-f7ee46d375a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.479702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-58138fab-7878-4492-ac1d-f7ee46d375a8 None None] MFA Rules not processed for user `3746c8dc35e046dcbbca4e7ffda46359`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:05.480870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-58138fab-7878-4492-ac1d-f7ee46d375a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:05.547907 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1189/2372] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:05.552839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbea71-d16d-4ec0-851f-865a58e63e5c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.614402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbea71-d16d-4ec0-851f-865a58e63e5c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.620248 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3bb03-c402-430d-a327-d58c583a963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.620934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3bb03-c402-430d-a327-d58c583a963f admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.621354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3bb03-c402-430d-a327-d58c583a963f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.621635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3bb03-c402-430d-a327-d58c583a963f admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.622469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3bb03-c402-430d-a327-d58c583a963f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.624947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3bb03-c402-430d-a327-d58c583a963f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.655509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1184/2373] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:05 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:05.659755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58138fab-7878-4492-ac1d-f7ee46d375a8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.670334 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58138fab-7878-4492-ac1d-f7ee46d375a8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.729670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bbc34-936e-4bb7-8de0-d68f61e6e932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.730242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bbc34-936e-4bb7-8de0-d68f61e6e932 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.730526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bbc34-936e-4bb7-8de0-d68f61e6e932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.730801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bbc34-936e-4bb7-8de0-d68f61e6e932 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.731471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bbc34-936e-4bb7-8de0-d68f61e6e932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.733437 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bbc34-936e-4bb7-8de0-d68f61e6e932 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.744783 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:05.759958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1190/2374] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:05 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:20:05.764014 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3bb03-c402-430d-a327-d58c583a963f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.776132 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3bb03-c402-430d-a327-d58c583a963f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.781232 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce2e7-5f6d-4f7c-92ac-29f7a256b85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.781870 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce2e7-5f6d-4f7c-92ac-29f7a256b85a admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.782149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce2e7-5f6d-4f7c-92ac-29f7a256b85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.782428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce2e7-5f6d-4f7c-92ac-29f7a256b85a admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.783148 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ce2e7-5f6d-4f7c-92ac-29f7a256b85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.784805 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ce2e7-5f6d-4f7c-92ac-29f7a256b85a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.799543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1185/2375] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:05.803562 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bbc34-936e-4bb7-8de0-d68f61e6e932 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.822812 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bbc34-936e-4bb7-8de0-d68f61e6e932 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.829877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd9ea-6dcd-4748-855b-52c5a6d718c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.830765 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd9ea-6dcd-4748-855b-52c5a6d718c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.831787 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd9ea-6dcd-4748-855b-52c5a6d718c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.831946 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fd9ea-6dcd-4748-855b-52c5a6d718c3 admin admin] PATH_INFO: `/v3/projects/ebef25bf0c6c4f20b24353cc37c9e90d/users/75e8539607824155a079e2b5fba3cea1/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.847608 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd9ea-6dcd-4748-855b-52c5a6d718c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebef25bf0c6c4f20b24353cc37c9e90d, user_id=75e8539607824155a079e2b5fba3cea1, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.850287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7fd9ea-6dcd-4748-855b-52c5a6d718c3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.864439 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1191/2376] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:05 2022] PUT /identity/v3/projects/ebef25bf0c6c4f20b24353cc37c9e90d/users/75e8539607824155a079e2b5fba3cea1/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:05.868354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce2e7-5f6d-4f7c-92ac-29f7a256b85a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.878745 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce2e7-5f6d-4f7c-92ac-29f7a256b85a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.945869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbfd93-2b40-41ed-a018-5a026863d958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.946824 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfd93-2b40-41ed-a018-5a026863d958 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.947362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfd93-2b40-41ed-a018-5a026863d958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.947828 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbfd93-2b40-41ed-a018-5a026863d958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:05.948944 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbfd93-2b40-41ed-a018-5a026863d958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:05.951838 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbfd93-2b40-41ed-a018-5a026863d958 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:05.971773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1186/2377] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:05.973612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd9ea-6dcd-4748-855b-52c5a6d718c3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:05.990368 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7fd9ea-6dcd-4748-855b-52c5a6d718c3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:05.997303 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:05.998315 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:05.998772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:05.999274 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] PATH_INFO: `/v3/projects/ebef25bf0c6c4f20b24353cc37c9e90d/users/75e8539607824155a079e2b5fba3cea1/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:06.002244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebef25bf0c6c4f20b24353cc37c9e90d, user_id=75e8539607824155a079e2b5fba3cea1, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:06.007691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:06.018697 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1192/2378] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:05 2022] PUT /identity/v3/projects/ebef25bf0c6c4f20b24353cc37c9e90d/users/75e8539607824155a079e2b5fba3cea1/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:06.029187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb5d04-01e3-49b8-938c-4b8aef8fe9f1 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:06.029477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb5d04-01e3-49b8-938c-4b8aef8fe9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:06.029755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb5d04-01e3-49b8-938c-4b8aef8fe9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:06.071850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-a8fb5d04-01e3-49b8-938c-4b8aef8fe9f1 None None] MFA Rules not processed for user `75e8539607824155a079e2b5fba3cea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:06.073251 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a8fb5d04-01e3-49b8-938c-4b8aef8fe9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:06.117297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:06.167789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1187/2379] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:06.172766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:06.177798 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:06.182078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb5d04-01e3-49b8-938c-4b8aef8fe9f1 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:06.183800 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a8fb5d04-01e3-49b8-938c-4b8aef8fe9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:06.248824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7568d-dd7b-430a-9b18-6f66ce6a0216 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:06.252970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db52b10-1245-42e6-88d1-7e27165a0bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:06.253618 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db52b10-1245-42e6-88d1-7e27165a0bdb service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:06.254037 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db52b10-1245-42e6-88d1-7e27165a0bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:06.254375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db52b10-1245-42e6-88d1-7e27165a0bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:06.256203 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb5d04-01e3-49b8-938c-4b8aef8fe9f1 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:06.256330 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4db52b10-1245-42e6-88d1-7e27165a0bdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:06.260157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b275615-8e5a-4fb9-a414-813a414cd2c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:06.261091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b275615-8e5a-4fb9-a414-813a414cd2c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:06.261444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b275615-8e5a-4fb9-a414-813a414cd2c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:06.261851 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b275615-8e5a-4fb9-a414-813a414cd2c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:06.273032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b275615-8e5a-4fb9-a414-813a414cd2c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:06.274760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b275615-8e5a-4fb9-a414-813a414cd2c8 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:06.285406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1188/2380] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:20:06.340175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db52b10-1245-42e6-88d1-7e27165a0bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:06.341688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db52b10-1245-42e6-88d1-7e27165a0bdb service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:06.353816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1193/2381] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:06.754714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b275615-8e5a-4fb9-a414-813a414cd2c8 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:06.829953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b275615-8e5a-4fb9-a414-813a414cd2c8 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:06.838202 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:06.838389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:06.838597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:06.838942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] PATH_INFO: `/v3/users/3746c8dc35e046dcbbca4e7ffda46359` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:06.840197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3746c8dc35e046dcbbca4e7ffda46359)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:06.842445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:06.926459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3746c8dc35e046dcbbca4e7ffda46359'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:06.940540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3746c8dc35e046dcbbca4e7ffda46359'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:06.941352 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3746c8dc35e046dcbbca4e7ffda46359'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:06.951427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3746c8dc35e046dcbbca4e7ffda46359'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:06.958681 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:06.962890 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3746c8dc35e046dcbbca4e7ffda46359'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:06.975720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1189/2382] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:06 2022] DELETE /identity/v3/users/3746c8dc35e046dcbbca4e7ffda46359 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:06.977882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff668ed-dbac-4641-8e49-c3dffab7d622 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:06.978388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff668ed-dbac-4641-8e49-c3dffab7d622 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:06.978668 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff668ed-dbac-4641-8e49-c3dffab7d622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:07.020423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1ff668ed-dbac-4641-8e49-c3dffab7d622 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:07.022096 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1ff668ed-dbac-4641-8e49-c3dffab7d622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:07.105641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1194/2383] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:07.118861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:07.120671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:07.183201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e140b5-d07c-412e-b869-02a34c0b539f admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:07.187386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb000f-97ec-480f-be3b-ef4706a82279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:07.188029 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb000f-97ec-480f-be3b-ef4706a82279 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:07.188319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb000f-97ec-480f-be3b-ef4706a82279 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:07.188711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb000f-97ec-480f-be3b-ef4706a82279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:07.198654 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb000f-97ec-480f-be3b-ef4706a82279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:07.200135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb000f-97ec-480f-be3b-ef4706a82279 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:07.210538 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1190/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:20:07.512169 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff668ed-dbac-4641-8e49-c3dffab7d622 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:07.513640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1ff668ed-dbac-4641-8e49-c3dffab7d622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:07.542524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff668ed-dbac-4641-8e49-c3dffab7d622 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:07.547503 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:07.548230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:07.548401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:07.548677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] PATH_INFO: `/v3/projects/1637b8c9d4364a578774bf9585f44ee2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:07.551864 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1637b8c9d4364a578774bf9585f44ee2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:07.553916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:07.556596 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:07.611699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1637b8c9d4364a578774bf9585f44ee2', 'request_id': 'req-843442ca-37cb-4f9e-92c6-908e5b90fdb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.620331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1637b8c9d4364a578774bf9585f44ee2', 'request_id': 'req-843442ca-37cb-4f9e-92c6-908e5b90fdb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.629532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1637b8c9d4364a578774bf9585f44ee2', 'request_id': 'req-843442ca-37cb-4f9e-92c6-908e5b90fdb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.640538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] The token cache is being invalidate because project 1637b8c9d4364a578774bf9585f44ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:07.640924 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.641623 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.643202 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1195/2385] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:07 2022] DELETE /identity/v3/projects/1637b8c9d4364a578774bf9585f44ee2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:07.647427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb000f-97ec-480f-be3b-ef4706a82279 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:07.657122 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb000f-97ec-480f-be3b-ef4706a82279 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:07.722091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:07.722730 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:07.723017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:07.723327 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] PATH_INFO: `/v3/users/75e8539607824155a079e2b5fba3cea1` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:07.724463 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e8539607824155a079e2b5fba3cea1)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:07.726406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:07.774378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e8539607824155a079e2b5fba3cea1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.796098 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e8539607824155a079e2b5fba3cea1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.796843 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e8539607824155a079e2b5fba3cea1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.805422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e8539607824155a079e2b5fba3cea1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.817180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0bead130-6deb-43d8-b1ce-83da9da4e67a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e8539607824155a079e2b5fba3cea1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:07.823120 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1191/2386] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:07 2022] DELETE /identity/v3/users/75e8539607824155a079e2b5fba3cea1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:08.126036 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.127744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:08.148168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843442ca-37cb-4f9e-92c6-908e5b90fdb8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.219235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.219968 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.220349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.220729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] PATH_INFO: `/v3/projects/ebef25bf0c6c4f20b24353cc37c9e90d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.222801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebef25bf0c6c4f20b24353cc37c9e90d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.225187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.285353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebef25bf0c6c4f20b24353cc37c9e90d', 'request_id': 'req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.293293 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebef25bf0c6c4f20b24353cc37c9e90d', 'request_id': 'req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.304069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebef25bf0c6c4f20b24353cc37c9e90d', 'request_id': 'req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.317342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] The token cache is being invalidate because project ebef25bf0c6c4f20b24353cc37c9e90d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:08.317739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.318473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.323180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1196/2387] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:08 2022] DELETE /identity/v3/projects/ebef25bf0c6c4f20b24353cc37c9e90d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:08.326247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022c5f9-31e2-4ce7-9858-21fc9ffda2de None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.326723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022c5f9-31e2-4ce7-9858-21fc9ffda2de None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.326852 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022c5f9-31e2-4ce7-9858-21fc9ffda2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.348334 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5022c5f9-31e2-4ce7-9858-21fc9ffda2de None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:08.349441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.349595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5022c5f9-31e2-4ce7-9858-21fc9ffda2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:08.350967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:08.369066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:08.392445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:08.449474 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1192/2388] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:08.463197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df48c90-6c3e-4ee8-ac9e-c28983c23796 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.463197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df48c90-6c3e-4ee8-ac9e-c28983c23796 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.463197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df48c90-6c3e-4ee8-ac9e-c28983c23796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.470953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa6a7-bc18-456d-9f01-b653ddf80adc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.476248 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.476892 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.477182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.477461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] PATH_INFO: `/v3/users/dab7d79197d04b5db9a381b9079a15ab` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.479485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dab7d79197d04b5db9a381b9079a15ab)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.481430 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.492934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-3df48c90-6c3e-4ee8-ac9e-c28983c23796 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:08.494160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-3df48c90-6c3e-4ee8-ac9e-c28983c23796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:08.505988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1193/2389] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:08.509974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df48c90-6c3e-4ee8-ac9e-c28983c23796 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.523017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df48c90-6c3e-4ee8-ac9e-c28983c23796 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.536709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.537634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.538309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.538595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.540323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:08.540648 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.543704 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.557335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dab7d79197d04b5db9a381b9079a15ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.562129 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1194/2390] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:20:08.568505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.581720 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dab7d79197d04b5db9a381b9079a15ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.582999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1984f-b1c7-4a5c-9021-09560717d4cf admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.583243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dab7d79197d04b5db9a381b9079a15ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.590406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.591147 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.591706 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.591970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.592673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.594051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dab7d79197d04b5db9a381b9079a15ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.594723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.595215 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5eb88222-06e8-4f1a-870c-4baef41bae86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dab7d79197d04b5db9a381b9079a15ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:08.610308 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1197/2391] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:08 2022] DELETE /identity/v3/users/dab7d79197d04b5db9a381b9079a15ab => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:08.616837 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d1d1-76b6-47ae-8d36-38bf4746cf46 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.617750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d1d1-76b6-47ae-8d36-38bf4746cf46 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.618258 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d1d1-76b6-47ae-8d36-38bf4746cf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.646277 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1195/2392] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:08 2022] POST /identity/v3/projects => generated 377 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:08.647437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.649733 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f256d1d1-76b6-47ae-8d36-38bf4746cf46 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:08.650125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:08.652873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f256d1d1-76b6-47ae-8d36-38bf4746cf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:08.735610 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34369b-1432-4f9e-abc6-41157e2bcdaf admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.741638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a2094-51f1-4ef9-911c-87f450eaf82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.742516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1198/2393] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:08.746149 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a2094-51f1-4ef9-911c-87f450eaf82b admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.746508 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a2094-51f1-4ef9-911c-87f450eaf82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.746796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a2094-51f1-4ef9-911c-87f450eaf82b admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.747509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a2094-51f1-4ef9-911c-87f450eaf82b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.750052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a2094-51f1-4ef9-911c-87f450eaf82b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.754522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256d1d1-76b6-47ae-8d36-38bf4746cf46 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.756053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f256d1d1-76b6-47ae-8d36-38bf4746cf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:08.781225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1196/2394] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:08 2022] POST /identity/v3/users => generated 289 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:08.783146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a2094-51f1-4ef9-911c-87f450eaf82b admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.794439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a2094-51f1-4ef9-911c-87f450eaf82b admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.813189 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b96c3-daf4-4111-8fb6-5418ea14a667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.814644 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b96c3-daf4-4111-8fb6-5418ea14a667 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.815830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b96c3-daf4-4111-8fb6-5418ea14a667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.816950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b96c3-daf4-4111-8fb6-5418ea14a667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.817786 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b96c3-daf4-4111-8fb6-5418ea14a667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.819792 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b96c3-daf4-4111-8fb6-5418ea14a667 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.832626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:08.834914 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1197/2395] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:08.838671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b96c3-daf4-4111-8fb6-5418ea14a667 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.850009 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256d1d1-76b6-47ae-8d36-38bf4746cf46 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.854524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36dbc5-f619-4efd-88f7-967b0614e5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.855188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36dbc5-f619-4efd-88f7-967b0614e5cf service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.855483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36dbc5-f619-4efd-88f7-967b0614e5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.855754 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36dbc5-f619-4efd-88f7-967b0614e5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.857305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b96c3-daf4-4111-8fb6-5418ea14a667 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.864329 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068491df-64be-470f-9c0b-f72f08ff8adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.865793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068491df-64be-470f-9c0b-f72f08ff8adb admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.866244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068491df-64be-470f-9c0b-f72f08ff8adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.866708 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068491df-64be-470f-9c0b-f72f08ff8adb admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.866833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36dbc5-f619-4efd-88f7-967b0614e5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.867649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068491df-64be-470f-9c0b-f72f08ff8adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.868266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36dbc5-f619-4efd-88f7-967b0614e5cf service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.869726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068491df-64be-470f-9c0b-f72f08ff8adb admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.881243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1199/2396] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:08.899772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1198/2397] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:08 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:20:08.901746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36dbc5-f619-4efd-88f7-967b0614e5cf service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:08.913985 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36dbc5-f619-4efd-88f7-967b0614e5cf service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:08.919014 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be40f15-08f8-42e2-9e64-80390f2042a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:08.919729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be40f15-08f8-42e2-9e64-80390f2042a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.920018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be40f15-08f8-42e2-9e64-80390f2042a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.920301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be40f15-08f8-42e2-9e64-80390f2042a3 admin admin] PATH_INFO: `/v3/projects/cb9a8411da9c4142b06dec430a31369a/users/6dd3f4eee3954b889fa95eaeb6e4b3d5/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:08.946020 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be40f15-08f8-42e2-9e64-80390f2042a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9a8411da9c4142b06dec430a31369a, user_id=6dd3f4eee3954b889fa95eaeb6e4b3d5, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:08.949531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be40f15-08f8-42e2-9e64-80390f2042a3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:08.971271 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1200/2398] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:08 2022] PUT /identity/v3/projects/cb9a8411da9c4142b06dec430a31369a/users/6dd3f4eee3954b889fa95eaeb6e4b3d5/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:08.983671 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe20e-0bc4-46ff-8f4a-740816bd1a56 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:08.983995 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe20e-0bc4-46ff-8f4a-740816bd1a56 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:08.985314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffe20e-0bc4-46ff-8f4a-740816bd1a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.079477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-15ffe20e-0bc4-46ff-8f4a-740816bd1a56 None None] MFA Rules not processed for user `6dd3f4eee3954b889fa95eaeb6e4b3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:09.081142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-15ffe20e-0bc4-46ff-8f4a-740816bd1a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:09.131550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:09.196510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1199/2399] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:09.202273 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be40f15-08f8-42e2-9e64-80390f2042a3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.251755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffe20e-0bc4-46ff-8f4a-740816bd1a56 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.254821 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-15ffe20e-0bc4-46ff-8f4a-740816bd1a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:09.286713 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be40f15-08f8-42e2-9e64-80390f2042a3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:09.293451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cef31-191f-4950-94a0-4716441d55eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:09.294074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cef31-191f-4950-94a0-4716441d55eb admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.294360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cef31-191f-4950-94a0-4716441d55eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.295297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cef31-191f-4950-94a0-4716441d55eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.295416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cef31-191f-4950-94a0-4716441d55eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:09.298205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cef31-191f-4950-94a0-4716441d55eb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:09.341986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1201/2400] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:09 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:09.345967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cef31-191f-4950-94a0-4716441d55eb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.353869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffe20e-0bc4-46ff-8f4a-740816bd1a56 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:09.426909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cef31-191f-4950-94a0-4716441d55eb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:09.428653 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:09.429467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.429750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.430021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] PATH_INFO: `/v3/projects/b636efc9fdd8414cb94268b52c1aea21` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.431269 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b636efc9fdd8414cb94268b52c1aea21)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:09.431865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a1a0d-a58a-42fc-8196-d8673fe6a7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:09.432482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a1a0d-a58a-42fc-8196-d8673fe6a7ea admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.432770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a1a0d-a58a-42fc-8196-d8673fe6a7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.433049 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1a1a0d-a58a-42fc-8196-d8673fe6a7ea admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.433398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:09.433694 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a1a0d-a58a-42fc-8196-d8673fe6a7ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:09.435688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1a1a0d-a58a-42fc-8196-d8673fe6a7ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:09.467023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1202/2401] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:09 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:09.467023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a1a0d-a58a-42fc-8196-d8673fe6a7ea admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.476974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1a1a0d-a58a-42fc-8196-d8673fe6a7ea admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:09.482942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb12872-8670-42be-92f5-5c9ddd526ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:09.483595 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb12872-8670-42be-92f5-5c9ddd526ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.483882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb12872-8670-42be-92f5-5c9ddd526ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.484184 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb12872-8670-42be-92f5-5c9ddd526ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.485387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb12872-8670-42be-92f5-5c9ddd526ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:09.486987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb12872-8670-42be-92f5-5c9ddd526ed0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:09.500130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1203/2402] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:09.506126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb12872-8670-42be-92f5-5c9ddd526ed0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.506815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b636efc9fdd8414cb94268b52c1aea21', 'request_id': 'req-2c9024c4-1b23-4481-bc61-cc36a97ba000'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.513583 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b636efc9fdd8414cb94268b52c1aea21', 'request_id': 'req-2c9024c4-1b23-4481-bc61-cc36a97ba000'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.519370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb12872-8670-42be-92f5-5c9ddd526ed0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:09.523765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b636efc9fdd8414cb94268b52c1aea21', 'request_id': 'req-2c9024c4-1b23-4481-bc61-cc36a97ba000'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.538466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] The token cache is being invalidate because project b636efc9fdd8414cb94268b52c1aea21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:09.538758 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.539517 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.541865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1200/2403] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:09 2022] DELETE /identity/v3/projects/b636efc9fdd8414cb94268b52c1aea21 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:09.544084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:09.546518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.547817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:09.571422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:09.594779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:09.595483 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.595964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.596260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] PATH_INFO: `/v3/projects/ebbc06174c4746e89a0b93666d931bc2/users/72f94d1fbe9c42d7998b382ea3f5e061/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.611291 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbc06174c4746e89a0b93666d931bc2, user_id=72f94d1fbe9c42d7998b382ea3f5e061, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:09.614830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:09.627187 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1204/2404] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:09 2022] PUT /identity/v3/projects/ebbc06174c4746e89a0b93666d931bc2/users/72f94d1fbe9c42d7998b382ea3f5e061/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:09.632198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.633972 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:09.634955 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9024c4-1b23-4481-bc61-cc36a97ba000 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:09.709543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:09.710791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.710791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.710791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] PATH_INFO: `/v3/users/fbec283587774e7c9d73b83347e9b678` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.711902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbec283587774e7c9d73b83347e9b678)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:09.713986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:09.725339 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc51f61-18f8-4471-9cd4-3dde6a223de7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:09.730071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a861d4-3bf6-4633-9879-f6d908820952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:09.730824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a861d4-3bf6-4633-9879-f6d908820952 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.731138 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a861d4-3bf6-4633-9879-f6d908820952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.731448 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a861d4-3bf6-4633-9879-f6d908820952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.732183 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a861d4-3bf6-4633-9879-f6d908820952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:09.733873 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a861d4-3bf6-4633-9879-f6d908820952 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:09.749071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1205/2405] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:20:09.754554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a861d4-3bf6-4633-9879-f6d908820952 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.766134 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a861d4-3bf6-4633-9879-f6d908820952 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:09.772085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c42b1-671f-4b2d-a70a-e06bdc53f4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:09.772986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c42b1-671f-4b2d-a70a-e06bdc53f4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.773474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c42b1-671f-4b2d-a70a-e06bdc53f4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.773755 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c42b1-671f-4b2d-a70a-e06bdc53f4f7 admin admin] PATH_INFO: `/v3/projects/ebbc06174c4746e89a0b93666d931bc2/users/72f94d1fbe9c42d7998b382ea3f5e061/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.775586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbec283587774e7c9d73b83347e9b678'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.775774 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c42b1-671f-4b2d-a70a-e06bdc53f4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbc06174c4746e89a0b93666d931bc2, user_id=72f94d1fbe9c42d7998b382ea3f5e061, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:09.778482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c42b1-671f-4b2d-a70a-e06bdc53f4f7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:09.789532 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbec283587774e7c9d73b83347e9b678'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.790261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbec283587774e7c9d73b83347e9b678'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.794125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1206/2406] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:09 2022] PUT /identity/v3/projects/ebbc06174c4746e89a0b93666d931bc2/users/72f94d1fbe9c42d7998b382ea3f5e061/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:09.799109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbec283587774e7c9d73b83347e9b678'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.805586 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6d96-f13d-4a2d-ac57-e975f707c76b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:09.805900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6d96-f13d-4a2d-ac57-e975f707c76b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:09.806174 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b6d96-f13d-4a2d-ac57-e975f707c76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:09.812703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbec283587774e7c9d73b83347e9b678'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:09.829605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1201/2407] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:09 2022] DELETE /identity/v3/users/fbec283587774e7c9d73b83347e9b678 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:09.842113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1c5b6d96-f13d-4a2d-ac57-e975f707c76b None None] MFA Rules not processed for user `72f94d1fbe9c42d7998b382ea3f5e061`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:09.843566 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1c5b6d96-f13d-4a2d-ac57-e975f707c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:09.932047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1207/2408] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:09.941278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:09.942793 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:09.988757 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:10.010444 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c5105-9a2e-4410-8096-000fe6afa545 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.014677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e87057-6105-423e-a180-571e530bd3a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.015834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e87057-6105-423e-a180-571e530bd3a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.015834 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e87057-6105-423e-a180-571e530bd3a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.016091 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e87057-6105-423e-a180-571e530bd3a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.017858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f7e87057-6105-423e-a180-571e530bd3a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:10.083619 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e87057-6105-423e-a180-571e530bd3a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.085073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e87057-6105-423e-a180-571e530bd3a7 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.095534 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1202/2409] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:10.155977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b6d96-f13d-4a2d-ac57-e975f707c76b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.157988 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1c5b6d96-f13d-4a2d-ac57-e975f707c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:10.185538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:10.236799 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b6d96-f13d-4a2d-ac57-e975f707c76b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.242281 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.242998 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.243406 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.243756 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] PATH_INFO: `/v3/projects/a97e6de2128542088e5f2049fa7926c3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.244966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a97e6de2128542088e5f2049fa7926c3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.247147 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.297594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a97e6de2128542088e5f2049fa7926c3', 'request_id': 'req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:10.305220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a97e6de2128542088e5f2049fa7926c3', 'request_id': 'req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:10.313160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a97e6de2128542088e5f2049fa7926c3', 'request_id': 'req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:10.325559 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] The token cache is being invalidate because project a97e6de2128542088e5f2049fa7926c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:10.325960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:10.326705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88b7ef9a-da6c-4b07-b5dd-aeaa65619a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:10.328222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1208/2410] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:10 2022] DELETE /identity/v3/projects/a97e6de2128542088e5f2049fa7926c3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:10.333953 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad157f4-5725-4efe-a448-3e400d5210e3 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.334244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad157f4-5725-4efe-a448-3e400d5210e3 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.334524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad157f4-5725-4efe-a448-3e400d5210e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.355577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-fad157f4-5725-4efe-a448-3e400d5210e3 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:10.356829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fad157f4-5725-4efe-a448-3e400d5210e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:10.427365 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1203/2411] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:10.434705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afdbf3-98d6-4921-a8b8-e0c9dd70e81e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.434997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afdbf3-98d6-4921-a8b8-e0c9dd70e81e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.435370 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afdbf3-98d6-4921-a8b8-e0c9dd70e81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.464177 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e2afdbf3-98d6-4921-a8b8-e0c9dd70e81e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:10.467650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e2afdbf3-98d6-4921-a8b8-e0c9dd70e81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:10.480041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1209/2412] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:10.484666 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad157f4-5725-4efe-a448-3e400d5210e3 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.494534 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad157f4-5725-4efe-a448-3e400d5210e3 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.499406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.500117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.500421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.500731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.501383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:10.501725 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.503686 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.516185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1204/2413] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:20:10.521141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afdbf3-98d6-4921-a8b8-e0c9dd70e81e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.530449 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afdbf3-98d6-4921-a8b8-e0c9dd70e81e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.536304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916918b-8513-4686-8ad5-ff2244ccf198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.537142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916918b-8513-4686-8ad5-ff2244ccf198 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.537443 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916918b-8513-4686-8ad5-ff2244ccf198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.537716 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916918b-8513-4686-8ad5-ff2244ccf198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.538340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916918b-8513-4686-8ad5-ff2244ccf198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.540253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916918b-8513-4686-8ad5-ff2244ccf198 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.567207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1210/2414] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:10 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:10.571543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.581739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddd174-4e5f-4d0c-99a6-e07196533829 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.637692 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78151c65-cf0b-4fb9-8923-0f95fbb07cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.638227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78151c65-cf0b-4fb9-8923-0f95fbb07cad admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.638509 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78151c65-cf0b-4fb9-8923-0f95fbb07cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.638832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78151c65-cf0b-4fb9-8923-0f95fbb07cad admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.639520 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78151c65-cf0b-4fb9-8923-0f95fbb07cad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.641495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78151c65-cf0b-4fb9-8923-0f95fbb07cad admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.662337 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1205/2415] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:10 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:10.666487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916918b-8513-4686-8ad5-ff2244ccf198 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.677168 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916918b-8513-4686-8ad5-ff2244ccf198 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.681706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee53e5-b767-4efa-9ab1-38472826d90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.682297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee53e5-b767-4efa-9ab1-38472826d90b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.682575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee53e5-b767-4efa-9ab1-38472826d90b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.682845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee53e5-b767-4efa-9ab1-38472826d90b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.683520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee53e5-b767-4efa-9ab1-38472826d90b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.685077 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ee53e5-b767-4efa-9ab1-38472826d90b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.695917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1211/2416] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:10.699840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78151c65-cf0b-4fb9-8923-0f95fbb07cad admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.710211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78151c65-cf0b-4fb9-8923-0f95fbb07cad admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.714767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb897b1d-ef2e-486a-ab31-4e1ed2f4bc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.715454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb897b1d-ef2e-486a-ab31-4e1ed2f4bc69 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.715737 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb897b1d-ef2e-486a-ab31-4e1ed2f4bc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.716013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb897b1d-ef2e-486a-ab31-4e1ed2f4bc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.716651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb897b1d-ef2e-486a-ab31-4e1ed2f4bc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.718254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb897b1d-ef2e-486a-ab31-4e1ed2f4bc69 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.720427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:10.730043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1206/2417] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:10.733897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee53e5-b767-4efa-9ab1-38472826d90b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.744429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee53e5-b767-4efa-9ab1-38472826d90b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.749228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ffb1f-de21-4cd6-b220-f523dafb90e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.749887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ffb1f-de21-4cd6-b220-f523dafb90e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.750162 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ffb1f-de21-4cd6-b220-f523dafb90e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.750438 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ffb1f-de21-4cd6-b220-f523dafb90e3 admin admin] PATH_INFO: `/v3/projects/7f9b39ae56ff481687af8bec44202fa5/users/9902545c0b1245789e13c1624b847f82/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.766351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ffb1f-de21-4cd6-b220-f523dafb90e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9b39ae56ff481687af8bec44202fa5, user_id=9902545c0b1245789e13c1624b847f82, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.769049 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ffb1f-de21-4cd6-b220-f523dafb90e3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.781773 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1212/2418] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:10 2022] PUT /identity/v3/projects/7f9b39ae56ff481687af8bec44202fa5/users/9902545c0b1245789e13c1624b847f82/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:10.793911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e19f26-7579-4684-998b-d04188dbbf71 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.794233 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e19f26-7579-4684-998b-d04188dbbf71 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.794513 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e19f26-7579-4684-998b-d04188dbbf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.826742 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-69e19f26-7579-4684-998b-d04188dbbf71 None None] MFA Rules not processed for user `9902545c0b1245789e13c1624b847f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:10.827970 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-69e19f26-7579-4684-998b-d04188dbbf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:10.894131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1207/2419] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:10.898593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ffb1f-de21-4cd6-b220-f523dafb90e3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:10.944516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:10.963400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ffb1f-de21-4cd6-b220-f523dafb90e3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:10.968039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf996ce-068e-4545-811f-80eca97b9551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:10.968642 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf996ce-068e-4545-811f-80eca97b9551 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:10.968923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf996ce-068e-4545-811f-80eca97b9551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:10.969199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf996ce-068e-4545-811f-80eca97b9551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:10.969836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf996ce-068e-4545-811f-80eca97b9551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:10.972494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf996ce-068e-4545-811f-80eca97b9551 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:10.998723 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1213/2420] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:10 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:11.002789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e19f26-7579-4684-998b-d04188dbbf71 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:11.013772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e19f26-7579-4684-998b-d04188dbbf71 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:11.069895 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a52dd-6eb9-4d5a-8989-3563c779e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:11.070505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a52dd-6eb9-4d5a-8989-3563c779e163 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.070791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a52dd-6eb9-4d5a-8989-3563c779e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.071187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a52dd-6eb9-4d5a-8989-3563c779e163 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.071841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a52dd-6eb9-4d5a-8989-3563c779e163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:11.073781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a52dd-6eb9-4d5a-8989-3563c779e163 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:11.095853 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1208/2421] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:11 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:11.100777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf996ce-068e-4545-811f-80eca97b9551 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:11.111046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf996ce-068e-4545-811f-80eca97b9551 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:11.116652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4a76f-32e2-4693-8342-b5a7fd5379bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:11.117277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a76f-32e2-4693-8342-b5a7fd5379bb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.117830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a76f-32e2-4693-8342-b5a7fd5379bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.118178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a76f-32e2-4693-8342-b5a7fd5379bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.118827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4a76f-32e2-4693-8342-b5a7fd5379bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:11.120492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d4a76f-32e2-4693-8342-b5a7fd5379bb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:11.131277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1214/2422] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:11.135559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a52dd-6eb9-4d5a-8989-3563c779e163 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:11.146443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a52dd-6eb9-4d5a-8989-3563c779e163 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:11.151106 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40a468-ad9d-4711-9d6c-81941a227c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:11.151764 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40a468-ad9d-4711-9d6c-81941a227c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.152048 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40a468-ad9d-4711-9d6c-81941a227c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.152330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40a468-ad9d-4711-9d6c-81941a227c9d admin admin] PATH_INFO: `/v3/projects/57b7566638744e458d0a2093ed73144c/users/5aa23dd768104ad69651e8c2184d661f/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.169557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40a468-ad9d-4711-9d6c-81941a227c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b7566638744e458d0a2093ed73144c, user_id=5aa23dd768104ad69651e8c2184d661f, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:11.173252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40a468-ad9d-4711-9d6c-81941a227c9d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:11.189746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1209/2423] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:11 2022] PUT /identity/v3/projects/57b7566638744e458d0a2093ed73144c/users/5aa23dd768104ad69651e8c2184d661f/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:11.194526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4a76f-32e2-4693-8342-b5a7fd5379bb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:11.205678 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4a76f-32e2-4693-8342-b5a7fd5379bb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:11.262347 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef623a-73ea-4491-ad06-b0332517840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:11.263041 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef623a-73ea-4491-ad06-b0332517840a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.263254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef623a-73ea-4491-ad06-b0332517840a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.263530 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef623a-73ea-4491-ad06-b0332517840a admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.264164 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef623a-73ea-4491-ad06-b0332517840a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:11.265770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef623a-73ea-4491-ad06-b0332517840a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:11.276091 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1215/2424] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:11.280235 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40a468-ad9d-4711-9d6c-81941a227c9d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:11.290878 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40a468-ad9d-4711-9d6c-81941a227c9d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:11.295747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:11.296399 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.296683 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.296958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] PATH_INFO: `/v3/projects/57b7566638744e458d0a2093ed73144c/users/5aa23dd768104ad69651e8c2184d661f/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.298894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57b7566638744e458d0a2093ed73144c, user_id=5aa23dd768104ad69651e8c2184d661f, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:11.301585 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:11.311803 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1210/2425] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:11 2022] PUT /identity/v3/projects/57b7566638744e458d0a2093ed73144c/users/5aa23dd768104ad69651e8c2184d661f/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:11.321771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294dbc-c2a4-4d21-8900-92193e8ab42f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.322060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294dbc-c2a4-4d21-8900-92193e8ab42f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.322336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e294dbc-c2a4-4d21-8900-92193e8ab42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.350918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8e294dbc-c2a4-4d21-8900-92193e8ab42f None None] MFA Rules not processed for user `5aa23dd768104ad69651e8c2184d661f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:11.352170 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e294dbc-c2a4-4d21-8900-92193e8ab42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:11.418047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1216/2426] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:11.429708 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:11.431529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:11.477491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:11.493387 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317edf53-3a12-47e1-ba3e-2a3bed3da934 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:11.497680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0df15-6412-4a1c-aa89-9c9ec1ba59be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:11.498293 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0df15-6412-4a1c-aa89-9c9ec1ba59be service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.498577 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0df15-6412-4a1c-aa89-9c9ec1ba59be service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.498856 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0df15-6412-4a1c-aa89-9c9ec1ba59be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.508289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0df15-6412-4a1c-aa89-9c9ec1ba59be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:11.509725 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0df15-6412-4a1c-aa89-9c9ec1ba59be service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:11.520416 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1211/2427] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:11.594006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e294dbc-c2a4-4d21-8900-92193e8ab42f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:11.604958 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e294dbc-c2a4-4d21-8900-92193e8ab42f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:11.608672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c0c02-0782-4af2-8396-09b1b3882191 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:11.609365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0c02-0782-4af2-8396-09b1b3882191 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:11.609648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0c02-0782-4af2-8396-09b1b3882191 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:11.609923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c0c02-0782-4af2-8396-09b1b3882191 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:11.648960 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:11.678997 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c0c02-0782-4af2-8396-09b1b3882191 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:11.680546 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c0c02-0782-4af2-8396-09b1b3882191 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:11.691977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1217/2428] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:11.965038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0df15-6412-4a1c-aa89-9c9ec1ba59be service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:12.027510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0df15-6412-4a1c-aa89-9c9ec1ba59be service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:12.032076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:12.032709 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:12.033005 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:12.033273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] PATH_INFO: `/v3/users/9902545c0b1245789e13c1624b847f82` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:12.034558 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9902545c0b1245789e13c1624b847f82)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:12.036529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:12.086303 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9902545c0b1245789e13c1624b847f82'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.099498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9902545c0b1245789e13c1624b847f82'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.100197 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9902545c0b1245789e13c1624b847f82'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.108798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9902545c0b1245789e13c1624b847f82'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.116978 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9902545c0b1245789e13c1624b847f82'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.123043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1212/2429] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:11 2022] DELETE /identity/v3/users/9902545c0b1245789e13c1624b847f82 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:12.127467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44f0c7-5f64-44f6-a8da-59b49b1c4966 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:12.128819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44f0c7-5f64-44f6-a8da-59b49b1c4966 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:12.128819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44f0c7-5f64-44f6-a8da-59b49b1c4966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:12.147701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7a44f0c7-5f64-44f6-a8da-59b49b1c4966 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:12.148898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7a44f0c7-5f64-44f6-a8da-59b49b1c4966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:12.206638 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1218/2430] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:12.216240 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:12.217927 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:12.276083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec56e9-72d8-4142-a478-9df4d18e07d0 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:12.279850 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d223bb2-521b-4045-825e-765d3aaebf83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:12.280461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d223bb2-521b-4045-825e-765d3aaebf83 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:12.280746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d223bb2-521b-4045-825e-765d3aaebf83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:12.281025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d223bb2-521b-4045-825e-765d3aaebf83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:12.290755 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d223bb2-521b-4045-825e-765d3aaebf83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:12.292227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d223bb2-521b-4045-825e-765d3aaebf83 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:12.302117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1213/2431] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:12.571787 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44f0c7-5f64-44f6-a8da-59b49b1c4966 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:12.573236 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7a44f0c7-5f64-44f6-a8da-59b49b1c4966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:12.585457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44f0c7-5f64-44f6-a8da-59b49b1c4966 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:12.590225 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:12.590853 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:12.591165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:12.591446 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] PATH_INFO: `/v3/projects/7f9b39ae56ff481687af8bec44202fa5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:12.592685 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9b39ae56ff481687af8bec44202fa5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:12.594593 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:12.651471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9b39ae56ff481687af8bec44202fa5', 'request_id': 'req-86f80ab7-f9e0-4271-8bce-7c85974faba6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.657534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9b39ae56ff481687af8bec44202fa5', 'request_id': 'req-86f80ab7-f9e0-4271-8bce-7c85974faba6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.667377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9b39ae56ff481687af8bec44202fa5', 'request_id': 'req-86f80ab7-f9e0-4271-8bce-7c85974faba6'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.677860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] The token cache is being invalidate because project 7f9b39ae56ff481687af8bec44202fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:12.678146 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.678764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.680235 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1219/2432] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:12 2022] DELETE /identity/v3/projects/7f9b39ae56ff481687af8bec44202fa5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:12.684169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d223bb2-521b-4045-825e-765d3aaebf83 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:12.690083 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:12.694354 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d223bb2-521b-4045-825e-765d3aaebf83 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:12.753456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:12.754027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:12.754309 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:12.754580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] PATH_INFO: `/v3/users/5aa23dd768104ad69651e8c2184d661f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:12.755797 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aa23dd768104ad69651e8c2184d661f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:12.757727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:12.811663 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aa23dd768104ad69651e8c2184d661f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.824587 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aa23dd768104ad69651e8c2184d661f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.825270 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aa23dd768104ad69651e8c2184d661f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.838719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aa23dd768104ad69651e8c2184d661f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.846074 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9ab40dd5-418a-481c-8ebf-729940c9dde7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aa23dd768104ad69651e8c2184d661f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:12.852464 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1214/2433] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:12 2022] DELETE /identity/v3/users/5aa23dd768104ad69651e8c2184d661f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:13.116515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.117973 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:13.133249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f80ab7-f9e0-4271-8bce-7c85974faba6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.188617 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.189241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.189520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.189794 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] PATH_INFO: `/v3/projects/57b7566638744e458d0a2093ed73144c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.191612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=57b7566638744e458d0a2093ed73144c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:13.193501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:13.194897 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:13.242650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57b7566638744e458d0a2093ed73144c', 'request_id': 'req-f4c96dce-25a8-4df8-987e-9a451c83ff7d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:13.248670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57b7566638744e458d0a2093ed73144c', 'request_id': 'req-f4c96dce-25a8-4df8-987e-9a451c83ff7d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:13.256457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57b7566638744e458d0a2093ed73144c', 'request_id': 'req-f4c96dce-25a8-4df8-987e-9a451c83ff7d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:13.267254 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] The token cache is being invalidate because project 57b7566638744e458d0a2093ed73144c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:13.267547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:13.268849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f4c96dce-25a8-4df8-987e-9a451c83ff7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:13.270793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1220/2434] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:13 2022] DELETE /identity/v3/projects/57b7566638744e458d0a2093ed73144c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:13.275606 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bb39b-2b93-4551-aecf-543a5cf9e8e9 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.275896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bb39b-2b93-4551-aecf-543a5cf9e8e9 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.276169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4bb39b-2b93-4551-aecf-543a5cf9e8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.296700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5c4bb39b-2b93-4551-aecf-543a5cf9e8e9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:13.297894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5c4bb39b-2b93-4551-aecf-543a5cf9e8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:13.359357 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1215/2435] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:13.368029 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9448-7caa-4ad8-a11a-4acf9878cbef None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.368414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9448-7caa-4ad8-a11a-4acf9878cbef None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.368806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff9448-7caa-4ad8-a11a-4acf9878cbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.388574 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3eff9448-7caa-4ad8-a11a-4acf9878cbef None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:13.389762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3eff9448-7caa-4ad8-a11a-4acf9878cbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:13.398475 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1221/2436] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:13.402662 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bb39b-2b93-4551-aecf-543a5cf9e8e9 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.412711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4bb39b-2b93-4551-aecf-543a5cf9e8e9 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.417225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.417816 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.418093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.418367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.419002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:13.419333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:13.420912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:13.432373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1216/2437] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:20:13.436701 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff9448-7caa-4ad8-a11a-4acf9878cbef None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.448237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff9448-7caa-4ad8-a11a-4acf9878cbef None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.452927 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdbbd2-03cf-46d8-a34c-44b71d82e213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.453527 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdbbd2-03cf-46d8-a34c-44b71d82e213 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.453808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdbbd2-03cf-46d8-a34c-44b71d82e213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.454082 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdbbd2-03cf-46d8-a34c-44b71d82e213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.454708 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdbbd2-03cf-46d8-a34c-44b71d82e213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:13.457480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdbbd2-03cf-46d8-a34c-44b71d82e213 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:13.485253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1222/2438] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:13 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:13.489439 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.499255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443e6a-8a33-4f48-b0ae-36b7e1ae9d8b admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.507187 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:13.561208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a64b7-fe49-4046-8ae0-e381c093afda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.561808 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a64b7-fe49-4046-8ae0-e381c093afda admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.562085 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a64b7-fe49-4046-8ae0-e381c093afda admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.562360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a64b7-fe49-4046-8ae0-e381c093afda admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.562985 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a64b7-fe49-4046-8ae0-e381c093afda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:13.564924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a64b7-fe49-4046-8ae0-e381c093afda admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:13.586225 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1217/2439] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:13 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:13.590374 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdbbd2-03cf-46d8-a34c-44b71d82e213 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.601957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdbbd2-03cf-46d8-a34c-44b71d82e213 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.606605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a78e0-2996-4a1d-978b-8c10d6209200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.607273 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a78e0-2996-4a1d-978b-8c10d6209200 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.607561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a78e0-2996-4a1d-978b-8c10d6209200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.607840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a78e0-2996-4a1d-978b-8c10d6209200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.608517 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a78e0-2996-4a1d-978b-8c10d6209200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:13.610115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a78e0-2996-4a1d-978b-8c10d6209200 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:13.620576 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1223/2440] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:13.624426 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a64b7-fe49-4046-8ae0-e381c093afda admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.634565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a64b7-fe49-4046-8ae0-e381c093afda admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.639425 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be31da0e-9b3e-4730-ac37-dbfc18a45d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.640025 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31da0e-9b3e-4730-ac37-dbfc18a45d34 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.640308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31da0e-9b3e-4730-ac37-dbfc18a45d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.640604 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be31da0e-9b3e-4730-ac37-dbfc18a45d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.641278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31da0e-9b3e-4730-ac37-dbfc18a45d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:13.642868 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be31da0e-9b3e-4730-ac37-dbfc18a45d34 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:13.654077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1218/2441] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:13.658232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a78e0-2996-4a1d-978b-8c10d6209200 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.669097 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a78e0-2996-4a1d-978b-8c10d6209200 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.673610 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391d1af-dbf6-4c1a-a136-c30a337bb8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.674249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391d1af-dbf6-4c1a-a136-c30a337bb8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.674528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391d1af-dbf6-4c1a-a136-c30a337bb8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.675310 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391d1af-dbf6-4c1a-a136-c30a337bb8a0 admin admin] PATH_INFO: `/v3/projects/48a14a27203a4573876ba17b8fc10c45/users/9d951a3d351748a1a7b741d5d060a783/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.689371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391d1af-dbf6-4c1a-a136-c30a337bb8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a14a27203a4573876ba17b8fc10c45, user_id=9d951a3d351748a1a7b741d5d060a783, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:13.692472 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2391d1af-dbf6-4c1a-a136-c30a337bb8a0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:13.705129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1224/2442] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:13 2022] PUT /identity/v3/projects/48a14a27203a4573876ba17b8fc10c45/users/9d951a3d351748a1a7b741d5d060a783/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:13.717040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273e4ac-b4a7-43eb-b914-a5f84424b351 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.717524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273e4ac-b4a7-43eb-b914-a5f84424b351 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.717625 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273e4ac-b4a7-43eb-b914-a5f84424b351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.746154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-5273e4ac-b4a7-43eb-b914-a5f84424b351 None None] MFA Rules not processed for user `9d951a3d351748a1a7b741d5d060a783`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:13.747395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-5273e4ac-b4a7-43eb-b914-a5f84424b351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:13.818987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1219/2443] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:13.823513 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391d1af-dbf6-4c1a-a136-c30a337bb8a0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.891830 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391d1af-dbf6-4c1a-a136-c30a337bb8a0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.897166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcea1e-b252-4fbc-9bfa-55bfd0096be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.897761 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcea1e-b252-4fbc-9bfa-55bfd0096be5 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.898040 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcea1e-b252-4fbc-9bfa-55bfd0096be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.898312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcea1e-b252-4fbc-9bfa-55bfd0096be5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.898934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcea1e-b252-4fbc-9bfa-55bfd0096be5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:13.900843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcea1e-b252-4fbc-9bfa-55bfd0096be5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:13.926638 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1225/2444] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:13 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:13.930690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273e4ac-b4a7-43eb-b914-a5f84424b351 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:13.941791 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273e4ac-b4a7-43eb-b914-a5f84424b351 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:13.997494 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cde707-8d0e-4ff4-b864-8717d1176e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:13.998094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cde707-8d0e-4ff4-b864-8717d1176e0a admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:13.998376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cde707-8d0e-4ff4-b864-8717d1176e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:13.998651 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cde707-8d0e-4ff4-b864-8717d1176e0a admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:13.999312 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cde707-8d0e-4ff4-b864-8717d1176e0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:14.001244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cde707-8d0e-4ff4-b864-8717d1176e0a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:14.023473 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1220/2445] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:13 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:14.027471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcea1e-b252-4fbc-9bfa-55bfd0096be5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:14.037660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcea1e-b252-4fbc-9bfa-55bfd0096be5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:14.042292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefae96a-9f09-40df-bd9b-4d52da427ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:14.042885 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae96a-9f09-40df-bd9b-4d52da427ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:14.043212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae96a-9f09-40df-bd9b-4d52da427ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:14.043490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefae96a-9f09-40df-bd9b-4d52da427ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:14.044135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefae96a-9f09-40df-bd9b-4d52da427ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:14.045709 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefae96a-9f09-40df-bd9b-4d52da427ce0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:14.051191 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:14.057322 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1226/2446] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:14.061596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cde707-8d0e-4ff4-b864-8717d1176e0a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:14.072466 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cde707-8d0e-4ff4-b864-8717d1176e0a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:14.077208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d01f06-efb1-43a2-aeb2-87bc7448ebdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:14.077875 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d01f06-efb1-43a2-aeb2-87bc7448ebdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:14.078525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d01f06-efb1-43a2-aeb2-87bc7448ebdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:14.078525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d01f06-efb1-43a2-aeb2-87bc7448ebdd admin admin] PATH_INFO: `/v3/projects/7c4a4d05d7784549a6315cee8cc11531/users/334084711ca943b896a5e7427a2e9c01/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:14.082717 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:14.096032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d01f06-efb1-43a2-aeb2-87bc7448ebdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4a4d05d7784549a6315cee8cc11531, user_id=334084711ca943b896a5e7427a2e9c01, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:14.098754 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d01f06-efb1-43a2-aeb2-87bc7448ebdd admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:14.111130 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1221/2447] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:14 2022] PUT /identity/v3/projects/7c4a4d05d7784549a6315cee8cc11531/users/334084711ca943b896a5e7427a2e9c01/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:14.115143 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefae96a-9f09-40df-bd9b-4d52da427ce0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:14.125658 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefae96a-9f09-40df-bd9b-4d52da427ce0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:14.182635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45416da5-67da-4cc1-a642-6500bfc92609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:14.183287 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416da5-67da-4cc1-a642-6500bfc92609 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:14.183552 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416da5-67da-4cc1-a642-6500bfc92609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:14.183826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45416da5-67da-4cc1-a642-6500bfc92609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:14.184465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45416da5-67da-4cc1-a642-6500bfc92609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:14.186222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45416da5-67da-4cc1-a642-6500bfc92609 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:14.196416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1227/2448] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:14.200779 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d01f06-efb1-43a2-aeb2-87bc7448ebdd admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:14.210947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d01f06-efb1-43a2-aeb2-87bc7448ebdd admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:14.215515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:14.216157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:14.216445 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:14.216724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] PATH_INFO: `/v3/projects/7c4a4d05d7784549a6315cee8cc11531/users/334084711ca943b896a5e7427a2e9c01/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:14.218656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c4a4d05d7784549a6315cee8cc11531, user_id=334084711ca943b896a5e7427a2e9c01, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:14.221385 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:14.231402 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1222/2449] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:14 2022] PUT /identity/v3/projects/7c4a4d05d7784549a6315cee8cc11531/users/334084711ca943b896a5e7427a2e9c01/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:14.242280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8ea7d-7ce9-49d4-ba44-aa977015fded None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:14.242573 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8ea7d-7ce9-49d4-ba44-aa977015fded None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:14.242855 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8ea7d-7ce9-49d4-ba44-aa977015fded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:14.270818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bbe8ea7d-7ce9-49d4-ba44-aa977015fded None None] MFA Rules not processed for user `334084711ca943b896a5e7427a2e9c01`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:14.272500 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bbe8ea7d-7ce9-49d4-ba44-aa977015fded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:14.348865 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1228/2450] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:14.356866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:14.358564 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:14.418869 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b3bbc-ec0e-4a39-8300-9e5028200979 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:14.422521 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc1a5-0efa-4740-a338-ea8bb95b704f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:14.423165 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc1a5-0efa-4740-a338-ea8bb95b704f service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:14.423452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc1a5-0efa-4740-a338-ea8bb95b704f service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:14.423729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc1a5-0efa-4740-a338-ea8bb95b704f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:14.480477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fc1a5-0efa-4740-a338-ea8bb95b704f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:14.481912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fc1a5-0efa-4740-a338-ea8bb95b704f service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:14.491483 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1223/2451] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:15.834039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8ea7d-7ce9-49d4-ba44-aa977015fded None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:15.835786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bbe8ea7d-7ce9-49d4-ba44-aa977015fded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:15.850888 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8ea7d-7ce9-49d4-ba44-aa977015fded None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:15.854920 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678974f-30be-42d9-8b56-a1f54daf24bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:15.855660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678974f-30be-42d9-8b56-a1f54daf24bd service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:15.855966 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678974f-30be-42d9-8b56-a1f54daf24bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:15.856246 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678974f-30be-42d9-8b56-a1f54daf24bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:15.858069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9678974f-30be-42d9-8b56-a1f54daf24bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:15.928233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678974f-30be-42d9-8b56-a1f54daf24bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:15.929719 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678974f-30be-42d9-8b56-a1f54daf24bd service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:15.941909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1229/2452] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:17.043543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc1a5-0efa-4740-a338-ea8bb95b704f service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:17.105703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc1a5-0efa-4740-a338-ea8bb95b704f service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:17.110208 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:17.110832 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:17.111143 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:17.111423 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] PATH_INFO: `/v3/users/9d951a3d351748a1a7b741d5d060a783` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:17.112537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d951a3d351748a1a7b741d5d060a783)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:17.114454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:17.144113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:17.164871 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d951a3d351748a1a7b741d5d060a783'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.177491 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d951a3d351748a1a7b741d5d060a783'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.178152 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d951a3d351748a1a7b741d5d060a783'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.185609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d951a3d351748a1a7b741d5d060a783'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.193448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d951a3d351748a1a7b741d5d060a783'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.199278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1224/2453] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:17 2022] DELETE /identity/v3/users/9d951a3d351748a1a7b741d5d060a783 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:17.203732 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25408-b54e-490a-a9de-18f9bb89d189 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:17.204222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25408-b54e-490a-a9de-18f9bb89d189 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:17.204326 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c25408-b54e-490a-a9de-18f9bb89d189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:17.222626 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c9c25408-b54e-490a-a9de-18f9bb89d189 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:17.223845 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c9c25408-b54e-490a-a9de-18f9bb89d189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:17.232746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:17.285772 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1230/2454] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:17.294848 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:17.296268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:17.359624 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74342dd9-799d-452b-ae7b-1ce66579da1c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:17.364447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4d774-2b9d-492a-83ae-69ba3c92957c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:17.365054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4d774-2b9d-492a-83ae-69ba3c92957c service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:17.365336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4d774-2b9d-492a-83ae-69ba3c92957c service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:17.365611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4d774-2b9d-492a-83ae-69ba3c92957c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:17.374635 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4d774-2b9d-492a-83ae-69ba3c92957c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:17.376174 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e4d774-2b9d-492a-83ae-69ba3c92957c service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:17.386711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1225/2455] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:17.668283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c25408-b54e-490a-a9de-18f9bb89d189 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:17.669780 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c9c25408-b54e-490a-a9de-18f9bb89d189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:17.685403 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c25408-b54e-490a-a9de-18f9bb89d189 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:17.691691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:17.692577 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:17.693031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:17.693471 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] PATH_INFO: `/v3/projects/48a14a27203a4573876ba17b8fc10c45` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:17.694870 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a14a27203a4573876ba17b8fc10c45)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:17.696814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:17.745531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a14a27203a4573876ba17b8fc10c45', 'request_id': 'req-4870e916-b543-4db8-8d35-b03754e6771d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.751638 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a14a27203a4573876ba17b8fc10c45', 'request_id': 'req-4870e916-b543-4db8-8d35-b03754e6771d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.759045 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a14a27203a4573876ba17b8fc10c45', 'request_id': 'req-4870e916-b543-4db8-8d35-b03754e6771d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.769877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] The token cache is being invalidate because project 48a14a27203a4573876ba17b8fc10c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:17.770165 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.770779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.772241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1231/2456] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:17 2022] DELETE /identity/v3/projects/48a14a27203a4573876ba17b8fc10c45 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:17.776284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4d774-2b9d-492a-83ae-69ba3c92957c service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:17.786099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e4d774-2b9d-492a-83ae-69ba3c92957c service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:17.842328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:17.842958 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:17.843276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:17.843555 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] PATH_INFO: `/v3/users/334084711ca943b896a5e7427a2e9c01` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:17.844672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=334084711ca943b896a5e7427a2e9c01)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:17.847447 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:17.896368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334084711ca943b896a5e7427a2e9c01'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.898288 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:17.910456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334084711ca943b896a5e7427a2e9c01'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.911144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334084711ca943b896a5e7427a2e9c01'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.921076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334084711ca943b896a5e7427a2e9c01'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.928988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-7412bfaa-6dc5-423c-b147-783afc3feb24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334084711ca943b896a5e7427a2e9c01'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:17.934899 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1226/2457] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:17 2022] DELETE /identity/v3/users/334084711ca943b896a5e7427a2e9c01 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:18.211898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:18.214230 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:18.228129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870e916-b543-4db8-8d35-b03754e6771d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:18.286529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:18.287196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.287480 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.287758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] PATH_INFO: `/v3/projects/7c4a4d05d7784549a6315cee8cc11531` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.288862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c4a4d05d7784549a6315cee8cc11531)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:18.290744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:18.336256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:18.347123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c4a4d05d7784549a6315cee8cc11531', 'request_id': 'req-6c400193-fd54-4e5e-974a-54d0dd31ea3a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:18.357915 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c4a4d05d7784549a6315cee8cc11531', 'request_id': 'req-6c400193-fd54-4e5e-974a-54d0dd31ea3a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:18.366840 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c4a4d05d7784549a6315cee8cc11531', 'request_id': 'req-6c400193-fd54-4e5e-974a-54d0dd31ea3a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:18.377149 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] The token cache is being invalidate because project 7c4a4d05d7784549a6315cee8cc11531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:18.377950 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:18.378670 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6c400193-fd54-4e5e-974a-54d0dd31ea3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:18.380182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1232/2458] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:18 2022] DELETE /identity/v3/projects/7c4a4d05d7784549a6315cee8cc11531 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:18.386478 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01193-8903-47a3-bb42-45a7752de4f9 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.386945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01193-8903-47a3-bb42-45a7752de4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.387092 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01193-8903-47a3-bb42-45a7752de4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.407481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-acc01193-8903-47a3-bb42-45a7752de4f9 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:18.408677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-acc01193-8903-47a3-bb42-45a7752de4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:18.468615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1227/2459] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:18.476615 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf991-97e6-4d88-bfa4-d89de2c2dc7a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.476911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf991-97e6-4d88-bfa4-d89de2c2dc7a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.477196 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cf991-97e6-4d88-bfa4-d89de2c2dc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.496508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-527cf991-97e6-4d88-bfa4-d89de2c2dc7a None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:18.497714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-527cf991-97e6-4d88-bfa4-d89de2c2dc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:18.506129 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1233/2460] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:18.510212 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc01193-8903-47a3-bb42-45a7752de4f9 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:18.520831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc01193-8903-47a3-bb42-45a7752de4f9 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:18.525448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:18.525987 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.526274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.526551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.527253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:18.527562 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:18.529162 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:18.541008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1228/2461] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:20:18.545186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf991-97e6-4d88-bfa4-d89de2c2dc7a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:18.555071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527cf991-97e6-4d88-bfa4-d89de2c2dc7a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:18.559650 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42070945-40b7-449c-9531-d9456d1eb227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:18.560241 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42070945-40b7-449c-9531-d9456d1eb227 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.560540 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42070945-40b7-449c-9531-d9456d1eb227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.560806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42070945-40b7-449c-9531-d9456d1eb227 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.561428 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42070945-40b7-449c-9531-d9456d1eb227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:18.563375 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42070945-40b7-449c-9531-d9456d1eb227 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:18.590814 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1234/2462] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:18 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:18.595078 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:18.605234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fcdeff-6006-48ea-9ed5-18916a5747ae admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:18.662053 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336974b-77c2-4163-b207-f70e05705e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:18.662658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336974b-77c2-4163-b207-f70e05705e2a admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.662945 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336974b-77c2-4163-b207-f70e05705e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.663252 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336974b-77c2-4163-b207-f70e05705e2a admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.663880 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336974b-77c2-4163-b207-f70e05705e2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:18.665803 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336974b-77c2-4163-b207-f70e05705e2a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:18.686954 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1229/2463] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:18 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:18.691862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42070945-40b7-449c-9531-d9456d1eb227 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:18.702136 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42070945-40b7-449c-9531-d9456d1eb227 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:18.707123 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b746d-357a-4e9d-b247-6ba590e9f664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:18.707735 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b746d-357a-4e9d-b247-6ba590e9f664 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.708019 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b746d-357a-4e9d-b247-6ba590e9f664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.708295 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b746d-357a-4e9d-b247-6ba590e9f664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.708965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b746d-357a-4e9d-b247-6ba590e9f664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:18.710533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b746d-357a-4e9d-b247-6ba590e9f664 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:18.720786 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1235/2464] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:18.725075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336974b-77c2-4163-b207-f70e05705e2a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:18.735199 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336974b-77c2-4163-b207-f70e05705e2a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:18.740276 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45522045-7eb7-462c-9e68-b1dfa1f2dbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:18.740871 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522045-7eb7-462c-9e68-b1dfa1f2dbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.741158 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522045-7eb7-462c-9e68-b1dfa1f2dbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.741437 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522045-7eb7-462c-9e68-b1dfa1f2dbd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.742080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45522045-7eb7-462c-9e68-b1dfa1f2dbd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:18.743908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45522045-7eb7-462c-9e68-b1dfa1f2dbd5 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:18.755340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1230/2465] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:18.759367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b746d-357a-4e9d-b247-6ba590e9f664 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:18.769651 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b746d-357a-4e9d-b247-6ba590e9f664 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:18.774231 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:18.774882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.775204 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.775539 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] PATH_INFO: `/v3/projects/6f6bba54ac0f49c0aa78b31b93bd54f6/users/bb9f380c57e3455cad6186d8146e8388/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.790522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6bba54ac0f49c0aa78b31b93bd54f6, user_id=bb9f380c57e3455cad6186d8146e8388, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:18.793273 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:18.806509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1236/2466] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:18 2022] PUT /identity/v3/projects/6f6bba54ac0f49c0aa78b31b93bd54f6/users/bb9f380c57e3455cad6186d8146e8388/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:18.873308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fbe50-4f6a-4199-977c-1c7570aeae7e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:18.873592 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fbe50-4f6a-4199-977c-1c7570aeae7e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:18.873865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7fbe50-4f6a-4199-977c-1c7570aeae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:18.902874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-4d7fbe50-4f6a-4199-977c-1c7570aeae7e None None] MFA Rules not processed for user `bb9f380c57e3455cad6186d8146e8388`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:18.904107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4d7fbe50-4f6a-4199-977c-1c7570aeae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:18.924746 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:18.970142 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1231/2467] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:18.982141 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:18.983863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:19.043549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac5a94-eddc-4ec8-bd1b-6b7a1866a044 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:19.047170 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a051b8-91ae-4982-b981-412e0c319602 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:19.047771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a051b8-91ae-4982-b981-412e0c319602 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:19.048053 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a051b8-91ae-4982-b981-412e0c319602 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:19.048327 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a051b8-91ae-4982-b981-412e0c319602 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:19.057359 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a051b8-91ae-4982-b981-412e0c319602 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:19.058558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a051b8-91ae-4982-b981-412e0c319602 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:19.069150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1237/2468] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:18 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:28.187652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fbe50-4f6a-4199-977c-1c7570aeae7e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:28.307292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7fbe50-4f6a-4199-977c-1c7570aeae7e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:28.311999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:28.312633 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:28.312924 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:28.313205 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] PATH_INFO: `/v3/users/bb9f380c57e3455cad6186d8146e8388` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:28.314360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9f380c57e3455cad6186d8146e8388)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:28.316408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:28.367286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9f380c57e3455cad6186d8146e8388'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:28.381006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9f380c57e3455cad6186d8146e8388'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:28.381705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9f380c57e3455cad6186d8146e8388'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:28.389760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9f380c57e3455cad6186d8146e8388'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:28.397672 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9f380c57e3455cad6186d8146e8388'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:28.400041 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:28.406052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1232/2469] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:28 2022] DELETE /identity/v3/users/bb9f380c57e3455cad6186d8146e8388 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:28.411112 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda7c9-93ab-4c64-9bbf-7f0346afa6fc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:28.411667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda7c9-93ab-4c64-9bbf-7f0346afa6fc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:28.411667 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eda7c9-93ab-4c64-9bbf-7f0346afa6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:28.468592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-90eda7c9-93ab-4c64-9bbf-7f0346afa6fc None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:28.469806 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-90eda7c9-93ab-4c64-9bbf-7f0346afa6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:28.509209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:28.531572 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1238/2470] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:28.541881 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:28.543330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:28.618104 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438fb79-589a-4556-9fe7-722ded7a8a1c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:28.621720 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:28.622325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:28.622607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:28.622882 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:28.633030 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:28.634456 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:28.644747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1233/2471] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:28.908247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda7c9-93ab-4c64-9bbf-7f0346afa6fc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:28.910978 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-90eda7c9-93ab-4c64-9bbf-7f0346afa6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:28.921123 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:28.922826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:28.924874 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eda7c9-93ab-4c64-9bbf-7f0346afa6fc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:28.930444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:28.931175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:28.931694 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:28.931981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] PATH_INFO: `/v3/projects/6f6bba54ac0f49c0aa78b31b93bd54f6` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:28.933903 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6bba54ac0f49c0aa78b31b93bd54f6)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:28.937711 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22dfc4-5319-436a-a93c-6571de1ba15b service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:28.941549 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:28.949947 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:28.950595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:28.950923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:28.951348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] PATH_INFO: `/v3/users/b7a68533d00f4ee4afe653569a6506b9` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:28.952355 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a68533d00f4ee4afe653569a6506b9)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:28.954331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:28.993983 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6bba54ac0f49c0aa78b31b93bd54f6', 'request_id': 'req-46f3636e-b5ae-475e-af83-e03f42474670'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.000189 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6bba54ac0f49c0aa78b31b93bd54f6', 'request_id': 'req-46f3636e-b5ae-475e-af83-e03f42474670'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.004733 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a68533d00f4ee4afe653569a6506b9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.008533 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6bba54ac0f49c0aa78b31b93bd54f6', 'request_id': 'req-46f3636e-b5ae-475e-af83-e03f42474670'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.017718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a68533d00f4ee4afe653569a6506b9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.018419 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a68533d00f4ee4afe653569a6506b9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.020416 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] The token cache is being invalidate because project 6f6bba54ac0f49c0aa78b31b93bd54f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:29.020778 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.021468 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-46f3636e-b5ae-475e-af83-e03f42474670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.023237 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1239/2472] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:28 2022] DELETE /identity/v3/projects/6f6bba54ac0f49c0aa78b31b93bd54f6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:29.026226 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a68533d00f4ee4afe653569a6506b9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.028632 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0444-568c-4d96-89a7-fa5443ad64a7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.028632 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0444-568c-4d96-89a7-fa5443ad64a7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.028939 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f0444-568c-4d96-89a7-fa5443ad64a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.034220 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4ba6eea2-713a-465c-8f34-c7e57fd9a204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a68533d00f4ee4afe653569a6506b9'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.041166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1234/2473] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:28 2022] DELETE /identity/v3/users/b7a68533d00f4ee4afe653569a6506b9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:29.045404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2fdec-2b9f-47de-93b9-e18584dbc8d1 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.046040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2fdec-2b9f-47de-93b9-e18584dbc8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.046321 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2fdec-2b9f-47de-93b9-e18584dbc8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.066173 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9ec2fdec-2b9f-47de-93b9-e18584dbc8d1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:29.067442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9ec2fdec-2b9f-47de-93b9-e18584dbc8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:29.080935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-eb3f0444-568c-4d96-89a7-fa5443ad64a7 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:29.082176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-eb3f0444-568c-4d96-89a7-fa5443ad64a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:29.122215 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:29.133118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1235/2474] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:29.143286 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2fdec-2b9f-47de-93b9-e18584dbc8d1 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.144818 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9ec2fdec-2b9f-47de-93b9-e18584dbc8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:29.145560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1240/2475] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:29.152934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6a02-a9c1-4074-aca7-842e5ebf0c16 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.153220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6a02-a9c1-4074-aca7-842e5ebf0c16 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.153496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d6a02-a9c1-4074-aca7-842e5ebf0c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.173118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-292d6a02-a9c1-4074-aca7-842e5ebf0c16 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:29.174490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-292d6a02-a9c1-4074-aca7-842e5ebf0c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:29.182304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1241/2476] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:20:29.186504 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d6a02-a9c1-4074-aca7-842e5ebf0c16 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.195841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d6a02-a9c1-4074-aca7-842e5ebf0c16 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.200678 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.201312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.201599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.201904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.202554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:29.202863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.204415 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2fdec-2b9f-47de-93b9-e18584dbc8d1 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.204902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.208950 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88820c28-6ca1-45df-b616-11e00009e134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.209559 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88820c28-6ca1-45df-b616-11e00009e134 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.209840 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88820c28-6ca1-45df-b616-11e00009e134 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.210115 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88820c28-6ca1-45df-b616-11e00009e134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.217222 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1242/2477] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 24 13:20:29.219323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88820c28-6ca1-45df-b616-11e00009e134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.220756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88820c28-6ca1-45df-b616-11e00009e134 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.222080 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.231155 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1236/2478] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:20:29.231795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7e0f1b-a36d-44e7-8552-bd3c0ad6ec6c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.236482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed0326-1048-4abc-b137-2faee47da387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.237220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed0326-1048-4abc-b137-2faee47da387 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.237492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed0326-1048-4abc-b137-2faee47da387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.237805 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed0326-1048-4abc-b137-2faee47da387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.238645 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed0326-1048-4abc-b137-2faee47da387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.240611 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed0326-1048-4abc-b137-2faee47da387 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.263208 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:29.274887 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1243/2479] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:29 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:20:29.276476 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88820c28-6ca1-45df-b616-11e00009e134 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.287965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88820c28-6ca1-45df-b616-11e00009e134 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.349714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd5052-8063-426e-8f1f-4a2f7d0f67ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.350325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5052-8063-426e-8f1f-4a2f7d0f67ab admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.350863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5052-8063-426e-8f1f-4a2f7d0f67ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.350863 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd5052-8063-426e-8f1f-4a2f7d0f67ab admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.351547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd5052-8063-426e-8f1f-4a2f7d0f67ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.353518 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd5052-8063-426e-8f1f-4a2f7d0f67ab admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.379600 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1237/2480] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:29 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:29.381065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed0326-1048-4abc-b137-2faee47da387 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.394211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed0326-1048-4abc-b137-2faee47da387 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.399635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caeb6d-e573-48d1-99f3-0782b6ec80a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.400332 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caeb6d-e573-48d1-99f3-0782b6ec80a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.400689 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caeb6d-e573-48d1-99f3-0782b6ec80a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.401034 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5caeb6d-e573-48d1-99f3-0782b6ec80a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.401757 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5caeb6d-e573-48d1-99f3-0782b6ec80a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.403458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5caeb6d-e573-48d1-99f3-0782b6ec80a1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.414835 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1244/2481] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:29.418795 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd5052-8063-426e-8f1f-4a2f7d0f67ab admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.428906 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd5052-8063-426e-8f1f-4a2f7d0f67ab admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.434037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97685429-9fb5-472b-9e22-f67247a6fe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.434772 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97685429-9fb5-472b-9e22-f67247a6fe92 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.435133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97685429-9fb5-472b-9e22-f67247a6fe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.435421 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97685429-9fb5-472b-9e22-f67247a6fe92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.436087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97685429-9fb5-472b-9e22-f67247a6fe92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.437896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97685429-9fb5-472b-9e22-f67247a6fe92 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.448652 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1238/2482] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:29.452834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caeb6d-e573-48d1-99f3-0782b6ec80a1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.464957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5caeb6d-e573-48d1-99f3-0782b6ec80a1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.470292 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.471777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.472163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.472499 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] PATH_INFO: `/v3/projects/e5f4729aa5f44fe08096bed3438e8f7e/users/1fe85edb1188403d84072d19128e07d5/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.490351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5f4729aa5f44fe08096bed3438e8f7e, user_id=1fe85edb1188403d84072d19128e07d5, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.493101 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.507969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1245/2483] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:29 2022] PUT /identity/v3/projects/e5f4729aa5f44fe08096bed3438e8f7e/users/1fe85edb1188403d84072d19128e07d5/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:29.513920 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178574a8-7b85-48de-b24d-c618617f6c9e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.514213 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178574a8-7b85-48de-b24d-c618617f6c9e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.514493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178574a8-7b85-48de-b24d-c618617f6c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.521482 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.523516 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:29.538299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-178574a8-7b85-48de-b24d-c618617f6c9e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:29.540550 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-178574a8-7b85-48de-b24d-c618617f6c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:29.602797 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381162b-b16c-4ca2-a1d0-ae24bb04f47a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.607515 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.608145 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.608429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.608707 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] PATH_INFO: `/v3/projects/49b426c946444937b56adfc4af5ea223` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.610084 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b426c946444937b56adfc4af5ea223)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.612076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.612340 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1239/2484] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:29.622696 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178574a8-7b85-48de-b24d-c618617f6c9e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:29.624184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-178574a8-7b85-48de-b24d-c618617f6c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:29.652435 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:29.661751 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b426c946444937b56adfc4af5ea223', 'request_id': 'req-19729f4d-eb3e-490e-9fc6-f905393000c9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.667802 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b426c946444937b56adfc4af5ea223', 'request_id': 'req-19729f4d-eb3e-490e-9fc6-f905393000c9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.675519 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b426c946444937b56adfc4af5ea223', 'request_id': 'req-19729f4d-eb3e-490e-9fc6-f905393000c9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.684708 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178574a8-7b85-48de-b24d-c618617f6c9e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:29.686763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] The token cache is being invalidate because project 49b426c946444937b56adfc4af5ea223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:29.687073 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.687727 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:29.689233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1246/2485] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:29 2022] DELETE /identity/v3/projects/49b426c946444937b56adfc4af5ea223 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:29.739761 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b4a38-fe26-4b84-89a8-07854d77be89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:29.740389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b4a38-fe26-4b84-89a8-07854d77be89 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:29.740673 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b4a38-fe26-4b84-89a8-07854d77be89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:29.740951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b4a38-fe26-4b84-89a8-07854d77be89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:29.742910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-1d4b4a38-fe26-4b84-89a8-07854d77be89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:29.812904 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b4a38-fe26-4b84-89a8-07854d77be89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:29.816186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b4a38-fe26-4b84-89a8-07854d77be89 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:29.827036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1240/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:32.227369 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:32.229102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:32.372543 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:32.418987 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19729f4d-eb3e-490e-9fc6-f905393000c9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:32.425477 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:32.425758 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:32.426098 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:32.426380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:32.428537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:32.442382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:32.444238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:32.468266 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1247/2487] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:34.401057 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf08b1-f583-40bd-8c22-34c8efa6e952 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:34.401565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf08b1-f583-40bd-8c22-34c8efa6e952 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:34.401682 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf08b1-f583-40bd-8c22-34c8efa6e952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:34.436012 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-efaf08b1-f583-40bd-8c22-34c8efa6e952 None None] MFA Rules not processed for user `1fe85edb1188403d84072d19128e07d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:34.437244 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-efaf08b1-f583-40bd-8c22-34c8efa6e952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:34.519769 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1241/2488] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:34.524336 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:34.525912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:34.540140 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83c926-7a19-427b-aafa-c8a1b783f74b service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:34.544766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f56496-8153-4b0c-84f8-8ef8e2b0dd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:34.545377 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f56496-8153-4b0c-84f8-8ef8e2b0dd88 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:34.545661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f56496-8153-4b0c-84f8-8ef8e2b0dd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:34.545935 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f56496-8153-4b0c-84f8-8ef8e2b0dd88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:34.546575 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f56496-8153-4b0c-84f8-8ef8e2b0dd88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:34.548505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f56496-8153-4b0c-84f8-8ef8e2b0dd88 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:34.585804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1248/2489] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:34 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:34.590075 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf08b1-f583-40bd-8c22-34c8efa6e952 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:34.601748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf08b1-f583-40bd-8c22-34c8efa6e952 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:34.616068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=100 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:34.664401 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd018b-7491-41cc-881e-eab74e1effb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:34.665043 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd018b-7491-41cc-881e-eab74e1effb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:34.665323 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd018b-7491-41cc-881e-eab74e1effb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:34.665591 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd018b-7491-41cc-881e-eab74e1effb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:34.666217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd018b-7491-41cc-881e-eab74e1effb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:34.668159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd018b-7491-41cc-881e-eab74e1effb3 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:34.691016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1242/2490] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:34 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:34.695659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f56496-8153-4b0c-84f8-8ef8e2b0dd88 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:34.707823 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f56496-8153-4b0c-84f8-8ef8e2b0dd88 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:34.712465 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fc2ff-162a-43ae-970c-870744639277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:34.713070 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fc2ff-162a-43ae-970c-870744639277 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:34.713354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fc2ff-162a-43ae-970c-870744639277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:34.713630 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fc2ff-162a-43ae-970c-870744639277 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:34.714406 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fc2ff-162a-43ae-970c-870744639277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:34.716066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fc2ff-162a-43ae-970c-870744639277 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:34.728827 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1249/2491] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:20:34.733206 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd018b-7491-41cc-881e-eab74e1effb3 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:34.744114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd018b-7491-41cc-881e-eab74e1effb3 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:34.749292 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059b01-6b00-41ed-8fb2-5639e355aa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:34.749974 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059b01-6b00-41ed-8fb2-5639e355aa11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:34.750256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059b01-6b00-41ed-8fb2-5639e355aa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:34.750531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3059b01-6b00-41ed-8fb2-5639e355aa11 admin admin] PATH_INFO: `/v3/projects/f4c3f792e1ae485fa9730e731b795550/users/8e513f79ed7b4a5292dd798e1776465a/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:34.766855 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3059b01-6b00-41ed-8fb2-5639e355aa11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c3f792e1ae485fa9730e731b795550, user_id=8e513f79ed7b4a5292dd798e1776465a, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:34.769614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3059b01-6b00-41ed-8fb2-5639e355aa11 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:34.782823 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1243/2492] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:34 2022] PUT /identity/v3/projects/f4c3f792e1ae485fa9730e731b795550/users/8e513f79ed7b4a5292dd798e1776465a/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:34.787816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fc2ff-162a-43ae-970c-870744639277 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:34.800551 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fc2ff-162a-43ae-970c-870744639277 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:34.861031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e583233b-8b7e-4aae-8552-89c81eec9dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:34.862367 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583233b-8b7e-4aae-8552-89c81eec9dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:34.862654 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583233b-8b7e-4aae-8552-89c81eec9dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:34.862923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583233b-8b7e-4aae-8552-89c81eec9dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:34.863594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583233b-8b7e-4aae-8552-89c81eec9dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:34.865170 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e583233b-8b7e-4aae-8552-89c81eec9dd1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:34.874739 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1250/2493] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:34.879647 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059b01-6b00-41ed-8fb2-5639e355aa11 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:34.889964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3059b01-6b00-41ed-8fb2-5639e355aa11 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:34.895185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:34.896530 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:34.896918 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:34.897195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] PATH_INFO: `/v3/projects/f4c3f792e1ae485fa9730e731b795550/users/8e513f79ed7b4a5292dd798e1776465a/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:34.899131 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c3f792e1ae485fa9730e731b795550, user_id=8e513f79ed7b4a5292dd798e1776465a, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:34.901773 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:34.912775 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1244/2494] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:34 2022] PUT /identity/v3/projects/f4c3f792e1ae485fa9730e731b795550/users/8e513f79ed7b4a5292dd798e1776465a/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:39.352583 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38d93a-53d7-464b-a460-ebbaf923bdce None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:39.353027 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38d93a-53d7-464b-a460-ebbaf923bdce None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:39.353137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38d93a-53d7-464b-a460-ebbaf923bdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:39.397752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ac38d93a-53d7-464b-a460-ebbaf923bdce None None] MFA Rules not processed for user `8e513f79ed7b4a5292dd798e1776465a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:39.399173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ac38d93a-53d7-464b-a460-ebbaf923bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:39.488641 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1251/2495] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:39.504815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:39.507560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:39.661037 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5897f7-cefd-4e04-bbe3-27c1ccb2a710 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:39.671801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43721c8-ad10-40ea-8dae-53db0fa52883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:39.672449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43721c8-ad10-40ea-8dae-53db0fa52883 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:39.672727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43721c8-ad10-40ea-8dae-53db0fa52883 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:39.673001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43721c8-ad10-40ea-8dae-53db0fa52883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:39.726688 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:39.747767 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43721c8-ad10-40ea-8dae-53db0fa52883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:39.749638 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43721c8-ad10-40ea-8dae-53db0fa52883 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:39.764044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1245/2496] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:20:42.550571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38d93a-53d7-464b-a460-ebbaf923bdce None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:42.553688 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ac38d93a-53d7-464b-a460-ebbaf923bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:42.571600 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:42.630349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38d93a-53d7-464b-a460-ebbaf923bdce None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:42.636848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:42.637702 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:42.638039 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:42.638531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] PATH_INFO: `/v3/users/6dd3f4eee3954b889fa95eaeb6e4b3d5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:42.640207 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dd3f4eee3954b889fa95eaeb6e4b3d5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:42.642233 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:42.708715 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dd3f4eee3954b889fa95eaeb6e4b3d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:42.722423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dd3f4eee3954b889fa95eaeb6e4b3d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:42.723467 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dd3f4eee3954b889fa95eaeb6e4b3d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:42.733212 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dd3f4eee3954b889fa95eaeb6e4b3d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:42.733834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dd3f4eee3954b889fa95eaeb6e4b3d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:42.750849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1252/2497] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:42 2022] DELETE /identity/v3/users/6dd3f4eee3954b889fa95eaeb6e4b3d5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:42.756442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab95a-6865-4b18-8878-71bebddc04ea None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:42.757310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab95a-6865-4b18-8878-71bebddc04ea None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:42.757762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfab95a-6865-4b18-8878-71bebddc04ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:42.779710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-dbfab95a-6865-4b18-8878-71bebddc04ea None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:42.780934 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-dbfab95a-6865-4b18-8878-71bebddc04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:42.856166 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1246/2498] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:42.866929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:42.868468 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:42.963371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba9459-a4d5-4f3b-a02f-cdfa25b8c955 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:42.967113 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef03a-8bfd-4971-8d9f-6cf94a4fb8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:42.967725 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef03a-8bfd-4971-8d9f-6cf94a4fb8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:42.968003 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef03a-8bfd-4971-8d9f-6cf94a4fb8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:42.968275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ef03a-8bfd-4971-8d9f-6cf94a4fb8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:42.978490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef03a-8bfd-4971-8d9f-6cf94a4fb8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:42.979965 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ef03a-8bfd-4971-8d9f-6cf94a4fb8a7 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:42.991150 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1253/2499] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:43.270217 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfab95a-6865-4b18-8878-71bebddc04ea None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:43.272061 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-dbfab95a-6865-4b18-8878-71bebddc04ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:43.285865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfab95a-6865-4b18-8878-71bebddc04ea None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:43.291008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:43.291747 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:43.292109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:43.292459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] PATH_INFO: `/v3/projects/cb9a8411da9c4142b06dec430a31369a` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:43.293676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9a8411da9c4142b06dec430a31369a)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:43.295694 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:43.356460 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9a8411da9c4142b06dec430a31369a', 'request_id': 'req-0419146b-55f0-4941-9f31-4e0d2bb867d7'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.363547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9a8411da9c4142b06dec430a31369a', 'request_id': 'req-0419146b-55f0-4941-9f31-4e0d2bb867d7'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.371026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9a8411da9c4142b06dec430a31369a', 'request_id': 'req-0419146b-55f0-4941-9f31-4e0d2bb867d7'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.381609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] The token cache is being invalidate because project cb9a8411da9c4142b06dec430a31369a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:43.381951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.382576 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.384176 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1247/2500] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:43 2022] DELETE /identity/v3/projects/cb9a8411da9c4142b06dec430a31369a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:43.388459 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef03a-8bfd-4971-8d9f-6cf94a4fb8a7 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:43.398767 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ef03a-8bfd-4971-8d9f-6cf94a4fb8a7 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:43.410538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:43.467738 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:43.468461 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:43.468746 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:43.469026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] PATH_INFO: `/v3/users/72f94d1fbe9c42d7998b382ea3f5e061` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:43.470256 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f94d1fbe9c42d7998b382ea3f5e061)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:43.472850 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:43.523777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f94d1fbe9c42d7998b382ea3f5e061'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.536298 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f94d1fbe9c42d7998b382ea3f5e061'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.537021 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f94d1fbe9c42d7998b382ea3f5e061'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.544224 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f94d1fbe9c42d7998b382ea3f5e061'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.544871 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e7dbda97-5375-45c7-a4cd-7a205b244435 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f94d1fbe9c42d7998b382ea3f5e061'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:43.557261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1254/2501] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:20:43 2022] DELETE /identity/v3/users/72f94d1fbe9c42d7998b382ea3f5e061 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:20:43.868325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:43.869801 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:43.883440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419146b-55f0-4941-9f31-4e0d2bb867d7 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:43.941988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:43.942618 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:43.942896 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:43.943205 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] PATH_INFO: `/v3/projects/ebbc06174c4746e89a0b93666d931bc2` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:43.944313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebbc06174c4746e89a0b93666d931bc2)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:43.946195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:43.953044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:43.999190 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebbc06174c4746e89a0b93666d931bc2', 'request_id': 'req-59ca9c0e-02d4-4606-abdf-5fc6080c5216'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:44.005515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebbc06174c4746e89a0b93666d931bc2', 'request_id': 'req-59ca9c0e-02d4-4606-abdf-5fc6080c5216'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:44.013504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebbc06174c4746e89a0b93666d931bc2', 'request_id': 'req-59ca9c0e-02d4-4606-abdf-5fc6080c5216'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:44.024892 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] The token cache is being invalidate because project ebbc06174c4746e89a0b93666d931bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:20:44.025180 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:44.025806 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-59ca9c0e-02d4-4606-abdf-5fc6080c5216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:20:44.027314 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1248/2502] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:20:43 2022] DELETE /identity/v3/projects/ebbc06174c4746e89a0b93666d931bc2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:44.032400 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a5470-1348-4546-acb9-3f795bae237e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.032696 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a5470-1348-4546-acb9-3f795bae237e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.033508 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a5470-1348-4546-acb9-3f795bae237e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.054581 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4e8a5470-1348-4546-acb9-3f795bae237e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:44.055863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4e8a5470-1348-4546-acb9-3f795bae237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:44.122969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1255/2503] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:44.131393 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6cf6c-14cb-4e38-bbef-b0e05d0e72e1 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.131680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6cf6c-14cb-4e38-bbef-b0e05d0e72e1 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.131956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6cf6c-14cb-4e38-bbef-b0e05d0e72e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.152553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-66d6cf6c-14cb-4e38-bbef-b0e05d0e72e1 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:44.153743 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-66d6cf6c-14cb-4e38-bbef-b0e05d0e72e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:44.162646 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1249/2504] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:44.166816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a5470-1348-4546-acb9-3f795bae237e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:44.176661 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a5470-1348-4546-acb9-3f795bae237e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:44.181186 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:44.181784 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.182069 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.182344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.182990 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:20:44.183348 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:44.184933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:44.201173 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1256/2505] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:20:44.206586 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6cf6c-14cb-4e38-bbef-b0e05d0e72e1 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:44.216153 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6cf6c-14cb-4e38-bbef-b0e05d0e72e1 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:44.221461 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2d446-27ac-4292-9c8c-824fa350aa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:44.222068 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d446-27ac-4292-9c8c-824fa350aa01 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.222349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d446-27ac-4292-9c8c-824fa350aa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.222653 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2d446-27ac-4292-9c8c-824fa350aa01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.223308 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2d446-27ac-4292-9c8c-824fa350aa01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:44.225204 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2d446-27ac-4292-9c8c-824fa350aa01 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:44.252967 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1250/2506] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:20:44 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:20:44.257273 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:44.268358 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f3984-6671-44f1-820e-cd42b46150f9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:44.288228 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:44.330904 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f3055-d10d-4c80-86f5-4eda818d81f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:44.331635 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f3055-d10d-4c80-86f5-4eda818d81f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.332071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f3055-d10d-4c80-86f5-4eda818d81f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.332445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f3055-d10d-4c80-86f5-4eda818d81f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.333160 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f3055-d10d-4c80-86f5-4eda818d81f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:44.335554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f3055-d10d-4c80-86f5-4eda818d81f3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:44.373547 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1257/2507] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:20:44 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:20:44.377607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2d446-27ac-4292-9c8c-824fa350aa01 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:44.393452 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2d446-27ac-4292-9c8c-824fa350aa01 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:44.398273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5425409-40df-45e6-9728-a8e4b16f2f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:44.398887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5425409-40df-45e6-9728-a8e4b16f2f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.399194 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5425409-40df-45e6-9728-a8e4b16f2f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.399467 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5425409-40df-45e6-9728-a8e4b16f2f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.400114 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5425409-40df-45e6-9728-a8e4b16f2f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:44.401699 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5425409-40df-45e6-9728-a8e4b16f2f79 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:44.413010 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1251/2508] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:44.417074 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f3055-d10d-4c80-86f5-4eda818d81f3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:44.427573 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f3055-d10d-4c80-86f5-4eda818d81f3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:44.432536 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f9ca7-e350-4a55-968e-9ae4c55c8f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:44.433243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f9ca7-e350-4a55-968e-9ae4c55c8f96 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.433555 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f9ca7-e350-4a55-968e-9ae4c55c8f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.433824 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f9ca7-e350-4a55-968e-9ae4c55c8f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.434466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f9ca7-e350-4a55-968e-9ae4c55c8f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:44.436223 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f9ca7-e350-4a55-968e-9ae4c55c8f96 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:44.453135 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1258/2509] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:20:44.458557 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5425409-40df-45e6-9728-a8e4b16f2f79 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:44.469969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5425409-40df-45e6-9728-a8e4b16f2f79 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:44.475858 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:44.476540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.476819 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.477093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] PATH_INFO: `/v3/projects/43d4b221c6774ac791b68e6d29766d09/users/4043a50708b54fe39d029263d9b59f7f/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.492427 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d4b221c6774ac791b68e6d29766d09, user_id=4043a50708b54fe39d029263d9b59f7f, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:44.495110 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:44.509328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1252/2510] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:20:44 2022] PUT /identity/v3/projects/43d4b221c6774ac791b68e6d29766d09/users/4043a50708b54fe39d029263d9b59f7f/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:20:44.521081 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0adb94-94f6-4827-9c52-d37857c57c9f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.521232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0adb94-94f6-4827-9c52-d37857c57c9f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.521833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0adb94-94f6-4827-9c52-d37857c57c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.573068 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7f0adb94-94f6-4827-9c52-d37857c57c9f None None] MFA Rules not processed for user `4043a50708b54fe39d029263d9b59f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:20:44.574360 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7f0adb94-94f6-4827-9c52-d37857c57c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:44.642182 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1259/2511] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:20:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:20:44.654304 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:20:44.656026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:20:44.720144 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906623ab-a243-4ee0-907b-d62ba2807f69 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:20:44.724356 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762e023-3ff6-4da6-9ac7-886343641a05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d0fc260e254d2d80c6927c52cd8a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:20:44.725054 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762e023-3ff6-4da6-9ac7-886343641a05 service glance] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:20:44.725405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762e023-3ff6-4da6-9ac7-886343641a05 service glance] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:20:44.725750 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762e023-3ff6-4da6-9ac7-886343641a05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:20:44.734366 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762e023-3ff6-4da6-9ac7-886343641a05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:20:44.735829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762e023-3ff6-4da6-9ac7-886343641a05 service glance] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:20:44.744948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1253/2512] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:20:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:20:54.639342 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:20:54.741032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:00.448198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e251bd-9b27-466b-85e9-5f937cf7e755 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:00.448714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e251bd-9b27-466b-85e9-5f937cf7e755 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:00.448714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e251bd-9b27-466b-85e9-5f937cf7e755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:00.468800 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d4e251bd-9b27-466b-85e9-5f937cf7e755 None None] MFA Rules not processed for user `1fe85edb1188403d84072d19128e07d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:00.470031 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d4e251bd-9b27-466b-85e9-5f937cf7e755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:00.525211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1260/2513] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:05.591200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762e023-3ff6-4da6-9ac7-886343641a05 service glance] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:05.654093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762e023-3ff6-4da6-9ac7-886343641a05 service glance] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:05.659096 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:05.659722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:05.660002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:05.660274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] PATH_INFO: `/v3/users/4043a50708b54fe39d029263d9b59f7f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:05.661389 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4043a50708b54fe39d029263d9b59f7f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:05.663429 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:05.709560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4043a50708b54fe39d029263d9b59f7f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:05.721601 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4043a50708b54fe39d029263d9b59f7f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:05.722261 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4043a50708b54fe39d029263d9b59f7f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:05.729087 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4043a50708b54fe39d029263d9b59f7f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:05.735849 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4043a50708b54fe39d029263d9b59f7f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:05.742547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1254/2514] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:21:05 2022] DELETE /identity/v3/users/4043a50708b54fe39d029263d9b59f7f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:05.747142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05aada-17d0-45fc-8b36-9393a4ad5ce8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:05.747640 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05aada-17d0-45fc-8b36-9393a4ad5ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:05.747744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05aada-17d0-45fc-8b36-9393a4ad5ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:05.767284 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7b05aada-17d0-45fc-8b36-9393a4ad5ce8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:05.768636 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7b05aada-17d0-45fc-8b36-9393a4ad5ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:05.827841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1261/2515] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:05.837777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:05.843923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:05.901095 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573f71b-0f0b-4888-a105-c43d8db4c6e6 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:05.904729 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6671c8-933f-45f7-a62a-42a37b0bb840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:05.905328 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6671c8-933f-45f7-a62a-42a37b0bb840 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:05.905609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6671c8-933f-45f7-a62a-42a37b0bb840 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:05.905883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6671c8-933f-45f7-a62a-42a37b0bb840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:05.914826 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6671c8-933f-45f7-a62a-42a37b0bb840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:05.916277 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6671c8-933f-45f7-a62a-42a37b0bb840 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:05.925898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1255/2516] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:06.273766 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05aada-17d0-45fc-8b36-9393a4ad5ce8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:06.275253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7b05aada-17d0-45fc-8b36-9393a4ad5ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:06.298007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05aada-17d0-45fc-8b36-9393a4ad5ce8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:06.305553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:06.306397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.306686 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.307155 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] PATH_INFO: `/v3/projects/43d4b221c6774ac791b68e6d29766d09` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.308531 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d4b221c6774ac791b68e6d29766d09)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:06.310655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:06.357335 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d4b221c6774ac791b68e6d29766d09', 'request_id': 'req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:06.363352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d4b221c6774ac791b68e6d29766d09', 'request_id': 'req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:06.371634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d4b221c6774ac791b68e6d29766d09', 'request_id': 'req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:06.381860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] The token cache is being invalidate because project 43d4b221c6774ac791b68e6d29766d09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:21:06.382153 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:06.382777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-419804b9-9fbe-4a74-bbb5-a9454bab9aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:06.384321 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1262/2517] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:21:06 2022] DELETE /identity/v3/projects/43d4b221c6774ac791b68e6d29766d09 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:06.389690 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98041a-4d73-4453-9b7d-b02b1f07dbd8 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.390008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98041a-4d73-4453-9b7d-b02b1f07dbd8 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.390294 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98041a-4d73-4453-9b7d-b02b1f07dbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.411498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-fd98041a-4d73-4453-9b7d-b02b1f07dbd8 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:06.412691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-fd98041a-4d73-4453-9b7d-b02b1f07dbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:06.429477 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:06.474367 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1256/2518] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:06.482057 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23728c-503e-4e9b-994a-befd1b90519f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.482352 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23728c-503e-4e9b-994a-befd1b90519f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.482637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23728c-503e-4e9b-994a-befd1b90519f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.502260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9c23728c-503e-4e9b-994a-befd1b90519f None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:06.503525 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9c23728c-503e-4e9b-994a-befd1b90519f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:06.511603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1263/2519] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:06.515722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98041a-4d73-4453-9b7d-b02b1f07dbd8 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:06.525802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98041a-4d73-4453-9b7d-b02b1f07dbd8 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:06.530345 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:06.530943 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.531264 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.531546 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.532191 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:21:06.532499 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:06.534794 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:06.546207 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1257/2520] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:21:06.550619 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23728c-503e-4e9b-994a-befd1b90519f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:06.559729 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23728c-503e-4e9b-994a-befd1b90519f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:06.564560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d694-5c2f-4643-b833-5bd77d1a5b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:06.565825 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d694-5c2f-4643-b833-5bd77d1a5b8e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.566111 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d694-5c2f-4643-b833-5bd77d1a5b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.566387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929d694-5c2f-4643-b833-5bd77d1a5b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.567035 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d694-5c2f-4643-b833-5bd77d1a5b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:06.569000 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0929d694-5c2f-4643-b833-5bd77d1a5b8e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:06.590486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:06.601300 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1264/2521] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:21:06 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:06.602593 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:06.612724 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6e14d-4962-4f0a-9bbf-64ca8831401e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:06.667839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8c10e-c463-43a9-ac45-04b16e206a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:06.668438 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c10e-c463-43a9-ac45-04b16e206a5d admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.668716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c10e-c463-43a9-ac45-04b16e206a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.668988 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c10e-c463-43a9-ac45-04b16e206a5d admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.669615 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8c10e-c463-43a9-ac45-04b16e206a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:06.671595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8c10e-c463-43a9-ac45-04b16e206a5d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:06.693297 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1258/2522] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:21:06 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:06.697401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d694-5c2f-4643-b833-5bd77d1a5b8e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:06.708390 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929d694-5c2f-4643-b833-5bd77d1a5b8e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:06.712964 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df07c8-9f65-4947-aaa9-af3bd67691b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:06.713557 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df07c8-9f65-4947-aaa9-af3bd67691b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.713841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df07c8-9f65-4947-aaa9-af3bd67691b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.714148 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df07c8-9f65-4947-aaa9-af3bd67691b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.714793 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df07c8-9f65-4947-aaa9-af3bd67691b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:06.716419 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df07c8-9f65-4947-aaa9-af3bd67691b1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:06.726244 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1265/2523] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:06.729959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8c10e-c463-43a9-ac45-04b16e206a5d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:06.739859 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8c10e-c463-43a9-ac45-04b16e206a5d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:06.744446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2e7bc-8dec-4cad-9ea1-85521963ce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:06.745039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2e7bc-8dec-4cad-9ea1-85521963ce39 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.745325 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2e7bc-8dec-4cad-9ea1-85521963ce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.745607 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2e7bc-8dec-4cad-9ea1-85521963ce39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.746256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2e7bc-8dec-4cad-9ea1-85521963ce39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:06.747913 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2e7bc-8dec-4cad-9ea1-85521963ce39 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:06.759017 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1259/2524] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:06.763441 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df07c8-9f65-4947-aaa9-af3bd67691b1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:06.773868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df07c8-9f65-4947-aaa9-af3bd67691b1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:06.778447 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171778-c04f-497e-b34d-c1560d35fae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:06.779119 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171778-c04f-497e-b34d-c1560d35fae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.779408 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171778-c04f-497e-b34d-c1560d35fae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.779690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5171778-c04f-497e-b34d-c1560d35fae6 admin admin] PATH_INFO: `/v3/projects/d53397758d5744a3a5fd645a48ae0a25/users/cc633cb238e2423cb751996661178d63/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.793898 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5171778-c04f-497e-b34d-c1560d35fae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53397758d5744a3a5fd645a48ae0a25, user_id=cc633cb238e2423cb751996661178d63, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:06.796589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5171778-c04f-497e-b34d-c1560d35fae6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:06.808808 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1266/2525] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:21:06 2022] PUT /identity/v3/projects/d53397758d5744a3a5fd645a48ae0a25/users/cc633cb238e2423cb751996661178d63/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:06.820021 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a950-e527-46b1-bf0d-557d3950ccd3 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.820310 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a950-e527-46b1-bf0d-557d3950ccd3 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.820585 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336a950-e527-46b1-bf0d-557d3950ccd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.848119 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-9336a950-e527-46b1-bf0d-557d3950ccd3 None None] MFA Rules not processed for user `cc633cb238e2423cb751996661178d63`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:06.849313 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9336a950-e527-46b1-bf0d-557d3950ccd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:06.912817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1260/2526] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:06.917466 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171778-c04f-497e-b34d-c1560d35fae6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:06.977687 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5171778-c04f-497e-b34d-c1560d35fae6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:06.982317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456d13-871f-4bd5-b648-b2ac297594ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:06.982917 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456d13-871f-4bd5-b648-b2ac297594ae admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:06.983226 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456d13-871f-4bd5-b648-b2ac297594ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:06.983505 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3456d13-871f-4bd5-b648-b2ac297594ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:06.984163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3456d13-871f-4bd5-b648-b2ac297594ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:06.986051 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3456d13-871f-4bd5-b648-b2ac297594ae admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.013060 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1267/2527] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:21:06 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:07.017094 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336a950-e527-46b1-bf0d-557d3950ccd3 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.027739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336a950-e527-46b1-bf0d-557d3950ccd3 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.039975 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:07.089637 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89675cba-b7ed-4b2a-b563-aaa360c8ff92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.090242 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89675cba-b7ed-4b2a-b563-aaa360c8ff92 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.090519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89675cba-b7ed-4b2a-b563-aaa360c8ff92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.090789 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89675cba-b7ed-4b2a-b563-aaa360c8ff92 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.091455 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89675cba-b7ed-4b2a-b563-aaa360c8ff92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.093362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89675cba-b7ed-4b2a-b563-aaa360c8ff92 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.114691 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1261/2528] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:21:07 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:07.118605 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456d13-871f-4bd5-b648-b2ac297594ae admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.129104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3456d13-871f-4bd5-b648-b2ac297594ae admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.133669 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f97c-0a87-42c4-a358-b61f2d519901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.134259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f97c-0a87-42c4-a358-b61f2d519901 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.134560 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f97c-0a87-42c4-a358-b61f2d519901 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.134816 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f97c-0a87-42c4-a358-b61f2d519901 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.135558 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579f97c-0a87-42c4-a358-b61f2d519901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.137382 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579f97c-0a87-42c4-a358-b61f2d519901 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.149613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1268/2529] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:07.153341 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89675cba-b7ed-4b2a-b563-aaa360c8ff92 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.164050 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89675cba-b7ed-4b2a-b563-aaa360c8ff92 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.168548 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01933ed8-3f14-4b5f-ac42-fa3136583a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.169159 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01933ed8-3f14-4b5f-ac42-fa3136583a40 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.169441 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01933ed8-3f14-4b5f-ac42-fa3136583a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.169714 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01933ed8-3f14-4b5f-ac42-fa3136583a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.170349 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01933ed8-3f14-4b5f-ac42-fa3136583a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.172141 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01933ed8-3f14-4b5f-ac42-fa3136583a40 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.182125 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1262/2530] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:07.186085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f97c-0a87-42c4-a358-b61f2d519901 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.196280 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f97c-0a87-42c4-a358-b61f2d519901 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.200929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c489d-ce20-42d1-95f1-11859a36f88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.201571 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c489d-ce20-42d1-95f1-11859a36f88f admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.201856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c489d-ce20-42d1-95f1-11859a36f88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.202137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c489d-ce20-42d1-95f1-11859a36f88f admin admin] PATH_INFO: `/v3/projects/e4acae419df9464b8b59f70094b44514/users/54824a2c85c743d5bc36f13da361cc8f/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.217203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c489d-ce20-42d1-95f1-11859a36f88f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4acae419df9464b8b59f70094b44514, user_id=54824a2c85c743d5bc36f13da361cc8f, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.219909 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c489d-ce20-42d1-95f1-11859a36f88f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.233006 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1269/2531] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:21:07 2022] PUT /identity/v3/projects/e4acae419df9464b8b59f70094b44514/users/54824a2c85c743d5bc36f13da361cc8f/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:07.299989 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addbd6-0115-4e2e-8b3c-eeacbf98d1b6 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.300296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addbd6-0115-4e2e-8b3c-eeacbf98d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.300563 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3addbd6-0115-4e2e-8b3c-eeacbf98d1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.329066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-f3addbd6-0115-4e2e-8b3c-eeacbf98d1b6 None None] MFA Rules not processed for user `54824a2c85c743d5bc36f13da361cc8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:07.330278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f3addbd6-0115-4e2e-8b3c-eeacbf98d1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:07.400529 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1263/2532] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:07.404975 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c489d-ce20-42d1-95f1-11859a36f88f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.468313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c489d-ce20-42d1-95f1-11859a36f88f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.473529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502e098-45e4-497c-b597-b1dbf33c4580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.474154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e098-45e4-497c-b597-b1dbf33c4580 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.474444 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e098-45e4-497c-b597-b1dbf33c4580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.474724 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502e098-45e4-497c-b597-b1dbf33c4580 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.475434 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502e098-45e4-497c-b597-b1dbf33c4580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.477379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502e098-45e4-497c-b597-b1dbf33c4580 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.498023 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:07.505541 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1270/2533] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:21:07 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:07.509556 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3addbd6-0115-4e2e-8b3c-eeacbf98d1b6 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.521362 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3addbd6-0115-4e2e-8b3c-eeacbf98d1b6 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.576731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c1439-edf8-4634-a801-e2acc9006001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.577333 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c1439-edf8-4634-a801-e2acc9006001 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.577616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c1439-edf8-4634-a801-e2acc9006001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.577889 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656c1439-edf8-4634-a801-e2acc9006001 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.578507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c1439-edf8-4634-a801-e2acc9006001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.580449 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656c1439-edf8-4634-a801-e2acc9006001 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.602505 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1264/2534] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:21:07 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:07.606589 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502e098-45e4-497c-b597-b1dbf33c4580 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.616956 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502e098-45e4-497c-b597-b1dbf33c4580 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.621512 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947ea56-eb79-4a51-b4fb-cf1bab488662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.622107 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947ea56-eb79-4a51-b4fb-cf1bab488662 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.622393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947ea56-eb79-4a51-b4fb-cf1bab488662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.622672 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2947ea56-eb79-4a51-b4fb-cf1bab488662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.623353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947ea56-eb79-4a51-b4fb-cf1bab488662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.624986 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2947ea56-eb79-4a51-b4fb-cf1bab488662 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.636481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1271/2535] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:07.640575 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c1439-edf8-4634-a801-e2acc9006001 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.651413 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656c1439-edf8-4634-a801-e2acc9006001 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.656039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc68b0-4b22-4289-ad80-4de984792b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.656676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc68b0-4b22-4289-ad80-4de984792b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.656956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc68b0-4b22-4289-ad80-4de984792b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.657227 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc68b0-4b22-4289-ad80-4de984792b93 admin admin] PATH_INFO: `/v3/projects/d62d48582c87422b872400259fec576f/users/fea82a8f4b1b46ae8a10e8a5dca18d3e/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.669370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:07.672700 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc68b0-4b22-4289-ad80-4de984792b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62d48582c87422b872400259fec576f, user_id=fea82a8f4b1b46ae8a10e8a5dca18d3e, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.676597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc68b0-4b22-4289-ad80-4de984792b93 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.690073 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1265/2536] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:21:07 2022] PUT /identity/v3/projects/d62d48582c87422b872400259fec576f/users/fea82a8f4b1b46ae8a10e8a5dca18d3e/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:07.694819 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947ea56-eb79-4a51-b4fb-cf1bab488662 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.704764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2947ea56-eb79-4a51-b4fb-cf1bab488662 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.760379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159193e7-ff8b-46e0-84f3-203b46a3504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.761253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159193e7-ff8b-46e0-84f3-203b46a3504b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.761537 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159193e7-ff8b-46e0-84f3-203b46a3504b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.761818 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159193e7-ff8b-46e0-84f3-203b46a3504b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.762456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159193e7-ff8b-46e0-84f3-203b46a3504b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.764065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159193e7-ff8b-46e0-84f3-203b46a3504b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.774379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1272/2537] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:21:07.778468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc68b0-4b22-4289-ad80-4de984792b93 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.788710 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc68b0-4b22-4289-ad80-4de984792b93 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:07.793302 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:07.793948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.794234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.794515 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] PATH_INFO: `/v3/projects/d62d48582c87422b872400259fec576f/users/fea82a8f4b1b46ae8a10e8a5dca18d3e/roles/11b74890c7484c6088dd78c106de8be4` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.796481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62d48582c87422b872400259fec576f, user_id=fea82a8f4b1b46ae8a10e8a5dca18d3e, role_id=11b74890c7484c6088dd78c106de8be4)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:07.799193 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:07.809991 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1266/2538] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:21:07 2022] PUT /identity/v3/projects/d62d48582c87422b872400259fec576f/users/fea82a8f4b1b46ae8a10e8a5dca18d3e/roles/11b74890c7484c6088dd78c106de8be4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:07.820914 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392285bf-6b04-4969-b7ca-a660d9ab8fb6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:07.821205 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392285bf-6b04-4969-b7ca-a660d9ab8fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:07.821484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392285bf-6b04-4969-b7ca-a660d9ab8fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:07.857115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-392285bf-6b04-4969-b7ca-a660d9ab8fb6 None None] MFA Rules not processed for user `fea82a8f4b1b46ae8a10e8a5dca18d3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:07.858340 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-392285bf-6b04-4969-b7ca-a660d9ab8fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:07.927992 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1273/2539] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:07.939395 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:07.941164 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:08.020027 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff003277-367e-4968-b968-795fdfdc3e71 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:08.023643 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a207d68-1f00-47ff-829e-25cef1ffc5b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:08.024240 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a207d68-1f00-47ff-829e-25cef1ffc5b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:08.024524 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a207d68-1f00-47ff-829e-25cef1ffc5b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:08.024796 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a207d68-1f00-47ff-829e-25cef1ffc5b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:08.080682 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a207d68-1f00-47ff-829e-25cef1ffc5b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:08.082765 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a207d68-1f00-47ff-829e-25cef1ffc5b5 service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:08.091602 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1267/2540] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:09.591532 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392285bf-6b04-4969-b7ca-a660d9ab8fb6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:09.592975 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-392285bf-6b04-4969-b7ca-a660d9ab8fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:09.605539 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392285bf-6b04-4969-b7ca-a660d9ab8fb6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:09.609087 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b96d0-404d-4e22-b362-b5b31f85e5eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:09.609687 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b96d0-404d-4e22-b362-b5b31f85e5eb service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:09.609974 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b96d0-404d-4e22-b362-b5b31f85e5eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:09.610249 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273b96d0-404d-4e22-b362-b5b31f85e5eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:09.667426 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b96d0-404d-4e22-b362-b5b31f85e5eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:09.668868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273b96d0-404d-4e22-b362-b5b31f85e5eb service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:09.678127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1274/2541] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:10.211032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a207d68-1f00-47ff-829e-25cef1ffc5b5 service cinder] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:10.219769 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a207d68-1f00-47ff-829e-25cef1ffc5b5 service cinder] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:10.224108 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d9b8f8-ce72-48f8-a68d-8bf1d8aa9c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:10.224707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b8f8-ce72-48f8-a68d-8bf1d8aa9c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:10.224990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b8f8-ce72-48f8-a68d-8bf1d8aa9c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:10.225268 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d9b8f8-ce72-48f8-a68d-8bf1d8aa9c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:10.234081 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9b8f8-ce72-48f8-a68d-8bf1d8aa9c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:10.235551 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d9b8f8-ce72-48f8-a68d-8bf1d8aa9c8d service cinder] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:10.244902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1268/2542] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:21:14.113764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b96d0-404d-4e22-b362-b5b31f85e5eb service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:14.123646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:14.177538 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273b96d0-404d-4e22-b362-b5b31f85e5eb service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:14.182070 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:14.182691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:14.182970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:14.183282 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] PATH_INFO: `/v3/users/cc633cb238e2423cb751996661178d63` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:14.184387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc633cb238e2423cb751996661178d63)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:14.186488 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:14.232804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc633cb238e2423cb751996661178d63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.244809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc633cb238e2423cb751996661178d63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.245457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc633cb238e2423cb751996661178d63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.253219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc633cb238e2423cb751996661178d63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.254297 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc633cb238e2423cb751996661178d63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.266604 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1275/2543] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:21:14 2022] DELETE /identity/v3/users/cc633cb238e2423cb751996661178d63 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:14.272184 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c1af-6166-453c-a1be-4d3fcbdb0559 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:14.272678 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c1af-6166-453c-a1be-4d3fcbdb0559 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:14.272777 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958c1af-6166-453c-a1be-4d3fcbdb0559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:14.291342 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-7958c1af-6166-453c-a1be-4d3fcbdb0559 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:14.292547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7958c1af-6166-453c-a1be-4d3fcbdb0559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:14.346540 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:14.357368 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1269/2544] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:14.367609 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:14.369102 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:14.431535 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d60aa6-2190-4224-a297-15fc928b43b3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:14.435365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0aa7c7-9042-4aa1-93ba-b4441fa634d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:14.436468 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0aa7c7-9042-4aa1-93ba-b4441fa634d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:14.436468 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0aa7c7-9042-4aa1-93ba-b4441fa634d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:14.436705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0aa7c7-9042-4aa1-93ba-b4441fa634d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:14.448594 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0aa7c7-9042-4aa1-93ba-b4441fa634d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:14.450125 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0aa7c7-9042-4aa1-93ba-b4441fa634d2 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:14.461220 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1276/2545] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:14.735408 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958c1af-6166-453c-a1be-4d3fcbdb0559 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:14.736883 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-7958c1af-6166-453c-a1be-4d3fcbdb0559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:14.750229 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958c1af-6166-453c-a1be-4d3fcbdb0559 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:14.756192 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:14.756923 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:14.757214 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:14.757495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] PATH_INFO: `/v3/projects/d53397758d5744a3a5fd645a48ae0a25` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:14.758675 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d53397758d5744a3a5fd645a48ae0a25)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:14.760894 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:14.808622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd53397758d5744a3a5fd645a48ae0a25', 'request_id': 'req-f5aaa00e-14c6-48b2-a572-296e95abef47'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.814916 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd53397758d5744a3a5fd645a48ae0a25', 'request_id': 'req-f5aaa00e-14c6-48b2-a572-296e95abef47'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.824507 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd53397758d5744a3a5fd645a48ae0a25', 'request_id': 'req-f5aaa00e-14c6-48b2-a572-296e95abef47'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.835168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] The token cache is being invalidate because project d53397758d5744a3a5fd645a48ae0a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:21:14.835448 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.836066 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.837537 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1270/2546] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:21:14 2022] DELETE /identity/v3/projects/d53397758d5744a3a5fd645a48ae0a25 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:14.841752 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0aa7c7-9042-4aa1-93ba-b4441fa634d2 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:14.851442 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0aa7c7-9042-4aa1-93ba-b4441fa634d2 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:14.881759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:14.910863 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:14.911524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:14.911809 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:14.912086 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] PATH_INFO: `/v3/users/54824a2c85c743d5bc36f13da361cc8f` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:14.913206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] RBAC: Authorizing `identity:delete_user(user_id=54824a2c85c743d5bc36f13da361cc8f)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:14.915161 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:14.960607 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54824a2c85c743d5bc36f13da361cc8f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.973455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54824a2c85c743d5bc36f13da361cc8f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.974130 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54824a2c85c743d5bc36f13da361cc8f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.981316 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54824a2c85c743d5bc36f13da361cc8f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.981942 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54824a2c85c743d5bc36f13da361cc8f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:14.994836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1277/2547] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:21:14 2022] DELETE /identity/v3/users/54824a2c85c743d5bc36f13da361cc8f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:15.281997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:15.283504 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:15.304695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aaa00e-14c6-48b2-a572-296e95abef47 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:15.367622 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:15.368181 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:15.368469 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:15.368748 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] PATH_INFO: `/v3/projects/e4acae419df9464b8b59f70094b44514` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:15.369902 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4acae419df9464b8b59f70094b44514)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:15.371874 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:15.423278 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4acae419df9464b8b59f70094b44514', 'request_id': 'req-9c8754cf-e300-4026-92a4-654c649bf7e1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.429831 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4acae419df9464b8b59f70094b44514', 'request_id': 'req-9c8754cf-e300-4026-92a4-654c649bf7e1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.438163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4acae419df9464b8b59f70094b44514', 'request_id': 'req-9c8754cf-e300-4026-92a4-654c649bf7e1'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.449093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] The token cache is being invalidate because project e4acae419df9464b8b59f70094b44514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:21:15.449391 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.450036 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.451608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1271/2548] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:21:15 2022] DELETE /identity/v3/projects/e4acae419df9464b8b59f70094b44514 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:15.455714 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:15.458166 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:15.470996 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd1d7a-9b9e-4d3c-8858-563fff70961e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:15.528192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:15.528826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:15.529108 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:15.529380 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] PATH_INFO: `/v3/users/fea82a8f4b1b46ae8a10e8a5dca18d3e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:15.530522 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea82a8f4b1b46ae8a10e8a5dca18d3e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:15.532492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:15.581963 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea82a8f4b1b46ae8a10e8a5dca18d3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.586691 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:15.597365 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea82a8f4b1b46ae8a10e8a5dca18d3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.598309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea82a8f4b1b46ae8a10e8a5dca18d3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.605902 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea82a8f4b1b46ae8a10e8a5dca18d3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.606644 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d49c9169-4cf0-4cbd-be22-01691f987134 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea82a8f4b1b46ae8a10e8a5dca18d3e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:15.619579 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1278/2549] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:21:15 2022] DELETE /identity/v3/users/fea82a8f4b1b46ae8a10e8a5dca18d3e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:15.894186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:15.896034 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:15.910299 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8754cf-e300-4026-92a4-654c649bf7e1 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:15.919725 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:15.972360 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:15.972990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:15.973273 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:15.973547 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] PATH_INFO: `/v3/projects/d62d48582c87422b872400259fec576f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:15.974658 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62d48582c87422b872400259fec576f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:15.976596 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:16.022959 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62d48582c87422b872400259fec576f', 'request_id': 'req-b7b02946-b850-4a2f-9d41-4d1af79636f8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:16.029117 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62d48582c87422b872400259fec576f', 'request_id': 'req-b7b02946-b850-4a2f-9d41-4d1af79636f8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:16.036723 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62d48582c87422b872400259fec576f', 'request_id': 'req-b7b02946-b850-4a2f-9d41-4d1af79636f8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:16.047316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] The token cache is being invalidate because project d62d48582c87422b872400259fec576f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:21:16.047605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:16.048240 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-b7b02946-b850-4a2f-9d41-4d1af79636f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:16.050256 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1272/2550] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:21:15 2022] DELETE /identity/v3/projects/d62d48582c87422b872400259fec576f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:16.055211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8302a-fed8-4de3-995e-fbc833e1bc03 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.055501 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8302a-fed8-4de3-995e-fbc833e1bc03 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.055779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8302a-fed8-4de3-995e-fbc833e1bc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.075866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a5b8302a-fed8-4de3-995e-fbc833e1bc03 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:16.077062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a5b8302a-fed8-4de3-995e-fbc833e1bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:16.136004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1279/2551] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:16.143898 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db83af-2424-4ade-b8f2-4d10f7ac4f40 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.144103 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db83af-2424-4ade-b8f2-4d10f7ac4f40 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.144378 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db83af-2424-4ade-b8f2-4d10f7ac4f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.164127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e5db83af-2424-4ade-b8f2-4d10f7ac4f40 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:16.165331 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e5db83af-2424-4ade-b8f2-4d10f7ac4f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:16.173198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1273/2552] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:16.178127 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8302a-fed8-4de3-995e-fbc833e1bc03 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:16.187822 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8302a-fed8-4de3-995e-fbc833e1bc03 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:16.193050 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:16.193647 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.193930 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.194206 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.194860 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:21:16.195203 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:16.196801 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:16.208309 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1280/2553] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:21:16.212493 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db83af-2424-4ade-b8f2-4d10f7ac4f40 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:16.221739 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db83af-2424-4ade-b8f2-4d10f7ac4f40 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:16.226418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345bb87-dedf-42c6-80b4-7832a23b0a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:16.227013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345bb87-dedf-42c6-80b4-7832a23b0a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.227326 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345bb87-dedf-42c6-80b4-7832a23b0a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.227605 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345bb87-dedf-42c6-80b4-7832a23b0a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.228239 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345bb87-dedf-42c6-80b4-7832a23b0a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:16.230134 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345bb87-dedf-42c6-80b4-7832a23b0a35 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:16.258033 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1274/2554] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:21:16 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 24 13:21:16.262353 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:16.272524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195fa36d-c0a0-4570-91b0-f5e24afd9343 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:16.333790 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4cec0-a0d9-40ca-af65-b7cd89af843a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:16.334401 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4cec0-a0d9-40ca-af65-b7cd89af843a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.334687 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4cec0-a0d9-40ca-af65-b7cd89af843a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.334970 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4cec0-a0d9-40ca-af65-b7cd89af843a admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.335737 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4cec0-a0d9-40ca-af65-b7cd89af843a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:16.337655 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4cec0-a0d9-40ca-af65-b7cd89af843a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:16.358705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1281/2555] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:21:16 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:16.363411 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345bb87-dedf-42c6-80b4-7832a23b0a35 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:16.373498 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345bb87-dedf-42c6-80b4-7832a23b0a35 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:16.378542 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248b87a-e762-4089-8f4a-cfc3c82b35e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:16.379247 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248b87a-e762-4089-8f4a-cfc3c82b35e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.379535 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248b87a-e762-4089-8f4a-cfc3c82b35e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.379813 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248b87a-e762-4089-8f4a-cfc3c82b35e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.380454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248b87a-e762-4089-8f4a-cfc3c82b35e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:16.382052 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248b87a-e762-4089-8f4a-cfc3c82b35e2 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:16.395457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1275/2556] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:16.396556 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4cec0-a0d9-40ca-af65-b7cd89af843a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:16.407137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4cec0-a0d9-40ca-af65-b7cd89af843a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:16.412496 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e66f44-1974-45a7-bc90-1a6246218294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:16.413100 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66f44-1974-45a7-bc90-1a6246218294 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.413394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66f44-1974-45a7-bc90-1a6246218294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.413677 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e66f44-1974-45a7-bc90-1a6246218294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.414331 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e66f44-1974-45a7-bc90-1a6246218294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:16.416019 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e66f44-1974-45a7-bc90-1a6246218294 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:16.427463 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1282/2557] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 24 13:21:16.431726 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248b87a-e762-4089-8f4a-cfc3c82b35e2 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:16.441716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248b87a-e762-4089-8f4a-cfc3c82b35e2 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:16.446319 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:16.446965 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.447348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.447628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] PATH_INFO: `/v3/projects/090f0886835f483b9e3b98a6a11eb973/users/95a1ded05fe64df5a77cc0cdc0c03d9f/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.461886 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=090f0886835f483b9e3b98a6a11eb973, user_id=95a1ded05fe64df5a77cc0cdc0c03d9f, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:16.464567 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:16.478257 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1276/2558] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:21:16 2022] PUT /identity/v3/projects/090f0886835f483b9e3b98a6a11eb973/users/95a1ded05fe64df5a77cc0cdc0c03d9f/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:21:16.490268 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6507e1-c3b7-4f6d-83cc-90a89783221c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.490748 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6507e1-c3b7-4f6d-83cc-90a89783221c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.490882 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6507e1-c3b7-4f6d-83cc-90a89783221c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.518803 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1e6507e1-c3b7-4f6d-83cc-90a89783221c None None] MFA Rules not processed for user `95a1ded05fe64df5a77cc0cdc0c03d9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:16.520242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1e6507e1-c3b7-4f6d-83cc-90a89783221c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:16.543967 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:16.585849 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1283/2559] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:16.597788 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:16.599516 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:16.646566 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:16.662361 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97b238-3dc9-4fe4-a1f1-aafec77cdb11 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:16.666022 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c657a3c-26cb-410b-a525-11c82c2b213c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19d0fc260e254d2d80c6927c52cd8a37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:16.666626 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c657a3c-26cb-410b-a525-11c82c2b213c service glance] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:16.666908 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c657a3c-26cb-410b-a525-11c82c2b213c service glance] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:16.667224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c657a3c-26cb-410b-a525-11c82c2b213c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:16.676020 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c657a3c-26cb-410b-a525-11c82c2b213c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:16.677453 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c657a3c-26cb-410b-a525-11c82c2b213c service glance] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:16.686350 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1277/2560] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:18.265286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6507e1-c3b7-4f6d-83cc-90a89783221c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:18.266750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1e6507e1-c3b7-4f6d-83cc-90a89783221c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:18.328286 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6507e1-c3b7-4f6d-83cc-90a89783221c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:18.332912 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f089b5ea-fea0-4764-9f1a-623a4f9b3149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:18.333862 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089b5ea-fea0-4764-9f1a-623a4f9b3149 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:18.334210 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089b5ea-fea0-4764-9f1a-623a4f9b3149 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:18.334485 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f089b5ea-fea0-4764-9f1a-623a4f9b3149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:18.336706 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f089b5ea-fea0-4764-9f1a-623a4f9b3149 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:18.404921 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089b5ea-fea0-4764-9f1a-623a4f9b3149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:18.406392 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f089b5ea-fea0-4764-9f1a-623a4f9b3149 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:18.417126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1284/2561] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:21:26.682253 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=8 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:28.412239 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=66 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:40.340649 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c657a3c-26cb-410b-a525-11c82c2b213c service glance] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:40.404108 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c657a3c-26cb-410b-a525-11c82c2b213c service glance] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:40.408533 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:40.409146 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:40.409424 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:40.409695 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] PATH_INFO: `/v3/users/95a1ded05fe64df5a77cc0cdc0c03d9f` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:40.411283 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95a1ded05fe64df5a77cc0cdc0c03d9f)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:40.413198 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:40.458861 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95a1ded05fe64df5a77cc0cdc0c03d9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:40.470616 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95a1ded05fe64df5a77cc0cdc0c03d9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:40.471432 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95a1ded05fe64df5a77cc0cdc0c03d9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:40.479830 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95a1ded05fe64df5a77cc0cdc0c03d9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:40.486941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95a1ded05fe64df5a77cc0cdc0c03d9f'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:40.493040 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1278/2562] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:21:40 2022] DELETE /identity/v3/users/95a1ded05fe64df5a77cc0cdc0c03d9f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:40.497474 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ca397-1b0d-4c33-9a57-d069368d5764 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:40.497954 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ca397-1b0d-4c33-9a57-d069368d5764 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:40.498065 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ca397-1b0d-4c33-9a57-d069368d5764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:40.517104 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f49ca397-1b0d-4c33-9a57-d069368d5764 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:40.518317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f49ca397-1b0d-4c33-9a57-d069368d5764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:40.575345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1285/2563] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:40.586201 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:40.587565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:40.644608 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40d4db-e48c-411a-aa50-e931680b0524 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:40.649330 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f120241-139c-4def-8a90-b065e7d8f13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:40.649930 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f120241-139c-4def-8a90-b065e7d8f13d service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:40.650209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f120241-139c-4def-8a90-b065e7d8f13d service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:40.650481 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f120241-139c-4def-8a90-b065e7d8f13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:40.659599 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f120241-139c-4def-8a90-b065e7d8f13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:40.661246 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f120241-139c-4def-8a90-b065e7d8f13d service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:40.670468 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1279/2564] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:40.935216 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ca397-1b0d-4c33-9a57-d069368d5764 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:40.936665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f49ca397-1b0d-4c33-9a57-d069368d5764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:40.949969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ca397-1b0d-4c33-9a57-d069368d5764 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:40.955486 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:40.956126 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:40.956408 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:40.956687 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] PATH_INFO: `/v3/projects/090f0886835f483b9e3b98a6a11eb973` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:40.959242 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090f0886835f483b9e3b98a6a11eb973)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:40.961350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:41.009989 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090f0886835f483b9e3b98a6a11eb973', 'request_id': 'req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:41.016304 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090f0886835f483b9e3b98a6a11eb973', 'request_id': 'req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:41.024351 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090f0886835f483b9e3b98a6a11eb973', 'request_id': 'req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:41.034779 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] The token cache is being invalidate because project 090f0886835f483b9e3b98a6a11eb973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:21:41.035095 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:41.036076 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42ca9c21-e7d1-4275-aacc-d8c269d2ccd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:21:41.037277 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1286/2565] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:21:40 2022] DELETE /identity/v3/projects/090f0886835f483b9e3b98a6a11eb973 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:41.042113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9352890-ee86-4c76-b3a0-d6011abf0800 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.042406 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9352890-ee86-4c76-b3a0-d6011abf0800 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.042684 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9352890-ee86-4c76-b3a0-d6011abf0800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.062829 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d9352890-ee86-4c76-b3a0-d6011abf0800 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:41.064063 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d9352890-ee86-4c76-b3a0-d6011abf0800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:41.081656 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:41.125702 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1280/2566] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:41.133163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc160-2078-4d8d-9151-f9104b93f035 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.133452 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc160-2078-4d8d-9151-f9104b93f035 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.133731 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7cc160-2078-4d8d-9151-f9104b93f035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.153411 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ab7cc160-2078-4d8d-9151-f9104b93f035 None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:41.154613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ab7cc160-2078-4d8d-9151-f9104b93f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:41.163046 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1287/2567] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:41.167205 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9352890-ee86-4c76-b3a0-d6011abf0800 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:41.177581 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9352890-ee86-4c76-b3a0-d6011abf0800 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:41.182186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:41.182815 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.183200 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.183484 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] PATH_INFO: `/v3/domains` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.184135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81614) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 24 13:21:41.184446 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:41.186069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:41.197721 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1281/2568] 10.222.0.42 () {58 vars in 1153 bytes} [Mon Oct 24 13:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 24 13:21:41.201900 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cc160-2078-4d8d-9151-f9104b93f035 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:41.210989 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7cc160-2078-4d8d-9151-f9104b93f035 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:41.215602 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d25b1-7c38-40f4-974c-934b9f182db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:41.216188 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d25b1-7c38-40f4-974c-934b9f182db9 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.216469 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d25b1-7c38-40f4-974c-934b9f182db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.216744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d25b1-7c38-40f4-974c-934b9f182db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.217393 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d25b1-7c38-40f4-974c-934b9f182db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:41.219317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d25b1-7c38-40f4-974c-934b9f182db9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:41.246621 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1288/2569] 10.222.0.42 () {60 vars in 1153 bytes} [Mon Oct 24 13:21:41 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:41.250756 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:41.261071 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206e005-0659-457f-8562-fea08ceae45c admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:41.318732 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c55e2-f495-403e-9133-37452035dc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:41.319370 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c55e2-f495-403e-9133-37452035dc78 admin admin] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.319676 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c55e2-f495-403e-9133-37452035dc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.319948 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c55e2-f495-403e-9133-37452035dc78 admin admin] PATH_INFO: `/v3/users` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.320560 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c55e2-f495-403e-9133-37452035dc78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:41.322495 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c55e2-f495-403e-9133-37452035dc78 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:41.342787 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1282/2570] 10.222.0.42 () {60 vars in 1144 bytes} [Mon Oct 24 13:21:41 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 24 13:21:41.346826 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d25b1-7c38-40f4-974c-934b9f182db9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:41.357933 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d25b1-7c38-40f4-974c-934b9f182db9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:41.362612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43052c-b1ca-48cf-a116-7b927b1d3e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:41.363253 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43052c-b1ca-48cf-a116-7b927b1d3e37 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.364175 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43052c-b1ca-48cf-a116-7b927b1d3e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.364490 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43052c-b1ca-48cf-a116-7b927b1d3e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.365209 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43052c-b1ca-48cf-a116-7b927b1d3e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:41.366807 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43052c-b1ca-48cf-a116-7b927b1d3e37 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:41.376841 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1289/2571] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:41.381238 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c55e2-f495-403e-9133-37452035dc78 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:41.391209 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c55e2-f495-403e-9133-37452035dc78 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:41.396169 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa249e-aec6-4ee6-9818-39457e7e1e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:41.396760 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa249e-aec6-4ee6-9818-39457e7e1e34 admin admin] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.397039 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa249e-aec6-4ee6-9818-39457e7e1e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.397315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa249e-aec6-4ee6-9818-39457e7e1e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.397956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa249e-aec6-4ee6-9818-39457e7e1e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:41.399565 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa249e-aec6-4ee6-9818-39457e7e1e34 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:41.409976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1283/2572] 10.222.0.42 () {58 vars in 1122 bytes} [Mon Oct 24 13:21:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 24 13:21:41.414908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43052c-b1ca-48cf-a116-7b927b1d3e37 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:41.424710 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43052c-b1ca-48cf-a116-7b927b1d3e37 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:41.430414 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:41.431071 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.431372 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.431648 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] PATH_INFO: `/v3/projects/a827ef206ac2466b8b28f472bd1d4813/users/3f411b926c43411fa9b41fc40055d1e9/roles/54e93caca0fb4c59a01ae64a0ed3a610` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.446763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a827ef206ac2466b8b28f472bd1d4813, user_id=3f411b926c43411fa9b41fc40055d1e9, role_id=54e93caca0fb4c59a01ae64a0ed3a610)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:41.449440 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:41.462429 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1290/2573] 10.222.0.42 () {60 vars in 1483 bytes} [Mon Oct 24 13:21:41 2022] PUT /identity/v3/projects/a827ef206ac2466b8b28f472bd1d4813/users/3f411b926c43411fa9b41fc40055d1e9/roles/54e93caca0fb4c59a01ae64a0ed3a610 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:21:41.474718 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e26d9c-e97a-450a-9e74-008fffe7bf3b None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.475006 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e26d9c-e97a-450a-9e74-008fffe7bf3b None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.475306 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e26d9c-e97a-450a-9e74-008fffe7bf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.503170 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-82e26d9c-e97a-450a-9e74-008fffe7bf3b None None] MFA Rules not processed for user `3f411b926c43411fa9b41fc40055d1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:41.504817 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-82e26d9c-e97a-450a-9e74-008fffe7bf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:41.569595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1284/2574] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:41.580388 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:41.582137 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:41.603561 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:41.641999 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a2fd9-fe7b-4974-9aef-ee20a1084686 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:41.645829 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4e9013-ed49-408d-a350-d5ba65a48f01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee3f4d7b69f4851b9160f68a3937703', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:41.646425 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e9013-ed49-408d-a350-d5ba65a48f01 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:41.646705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e9013-ed49-408d-a350-d5ba65a48f01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:41.646977 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4e9013-ed49-408d-a350-d5ba65a48f01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:41.655042 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e9013-ed49-408d-a350-d5ba65a48f01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:41.656473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4e9013-ed49-408d-a350-d5ba65a48f01 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:41.665456 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1291/2575] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:21:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:21:43.029224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ad9a-753b-4e4e-83da-2823b2a6e1ac None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:43.029770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ad9a-753b-4e4e-83da-2823b2a6e1ac None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:43.029770 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7ad9a-753b-4e4e-83da-2823b2a6e1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:43.048964 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-e4b7ad9a-753b-4e4e-83da-2823b2a6e1ac None None] MFA Rules not processed for user `3f411b926c43411fa9b41fc40055d1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:43.050258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e4b7ad9a-753b-4e4e-83da-2823b2a6e1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:43.057580 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1285/2576] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:21:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:51.661598 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=17 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:53.043544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:21:56.994698 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a07a6-dd04-4ddc-9579-445d1f24f330 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:56.995283 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a07a6-dd04-4ddc-9579-445d1f24f330 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:56.995389 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a07a6-dd04-4ddc-9579-445d1f24f330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:57.025142 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-de5a07a6-dd04-4ddc-9579-445d1f24f330 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:57.026387 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-de5a07a6-dd04-4ddc-9579-445d1f24f330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:57.083509 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1292/2577] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:21:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:57.129407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7ad9a-753b-4e4e-83da-2823b2a6e1ac None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:57.130921 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-e4b7ad9a-753b-4e4e-83da-2823b2a6e1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:57.199833 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7ad9a-753b-4e4e-83da-2823b2a6e1ac None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:57.204790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77081e6-2d82-4932-a711-52970711b556 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:57.205397 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77081e6-2d82-4932-a711-52970711b556 service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:57.205680 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77081e6-2d82-4932-a711-52970711b556 service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:57.205962 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77081e6-2d82-4932-a711-52970711b556 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:57.214595 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77081e6-2d82-4932-a711-52970711b556 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:57.216282 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77081e6-2d82-4932-a711-52970711b556 service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:57.225126 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1286/2578] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:21:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:21:57.718460 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b525-8cc1-48e0-8154-030b28b91640 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:57.718750 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b525-8cc1-48e0-8154-030b28b91640 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:57.719032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411b525-8cc1-48e0-8154-030b28b91640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:57.738044 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d411b525-8cc1-48e0-8154-030b28b91640 None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:57.739267 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d411b525-8cc1-48e0-8154-030b28b91640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:57.746495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1293/2579] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 24 13:21:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:57.759026 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77081e6-2d82-4932-a711-52970711b556 service swift] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:57.769866 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77081e6-2d82-4932-a711-52970711b556 service swift] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:57.775482 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033006-1760-467f-b025-982a21515143 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:57.776076 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033006-1760-467f-b025-982a21515143 service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:57.776373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033006-1760-467f-b025-982a21515143 service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:57.776628 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033006-1760-467f-b025-982a21515143 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:57.784450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033006-1760-467f-b025-982a21515143 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:57.785999 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033006-1760-467f-b025-982a21515143 service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:57.794271 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1287/2580] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:21:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:21:58.018245 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749948c4-17e1-4c4a-b26b-b104cbdc56bd None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:58.018699 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749948c4-17e1-4c4a-b26b-b104cbdc56bd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:58.018828 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749948c4-17e1-4c4a-b26b-b104cbdc56bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:58.038554 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-749948c4-17e1-4c4a-b26b-b104cbdc56bd None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:58.039836 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-749948c4-17e1-4c4a-b26b-b104cbdc56bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:58.047590 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1294/2581] 10.222.0.42 () {58 vars in 1009 bytes} [Mon Oct 24 13:21:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:21:58.060113 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033006-1760-467f-b025-982a21515143 service swift] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:58.072002 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033006-1760-467f-b025-982a21515143 service swift] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:58.077109 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886621b-7821-4c30-adeb-c11e769ece2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:58.078049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886621b-7821-4c30-adeb-c11e769ece2a service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:58.078049 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886621b-7821-4c30-adeb-c11e769ece2a service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:58.078324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886621b-7821-4c30-adeb-c11e769ece2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:58.086664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886621b-7821-4c30-adeb-c11e769ece2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:58.088289 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886621b-7821-4c30-adeb-c11e769ece2a service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:58.096219 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1288/2582] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:21:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 24 13:21:58.417099 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80cab7-b617-4594-af23-c0fdf0de207c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:58.417385 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80cab7-b617-4594-af23-c0fdf0de207c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:58.417660 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80cab7-b617-4594-af23-c0fdf0de207c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:58.437957 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fd80cab7-b617-4594-af23-c0fdf0de207c None None] MFA Rules not processed for user `b2396a5c6002491f926326ee2d154f92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:21:58.439176 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fd80cab7-b617-4594-af23-c0fdf0de207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:21:58.446154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1295/2583] 10.222.0.42 () {58 vars in 967 bytes} [Mon Oct 24 13:21:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:21:58.458163 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886621b-7821-4c30-adeb-c11e769ece2a service swift] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:21:58.468951 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886621b-7821-4c30-adeb-c11e769ece2a service swift] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:21:58.473925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49a640-5f5b-453c-bd1d-4540de9e4c77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5a63131a456548c8a3a3e85dbeba5a48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:21:58.475016 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a640-5f5b-453c-bd1d-4540de9e4c77 service swift] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:21:58.475479 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a640-5f5b-453c-bd1d-4540de9e4c77 service swift] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:21:58.475588 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a640-5f5b-453c-bd1d-4540de9e4c77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:21:58.483543 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49a640-5f5b-453c-bd1d-4540de9e4c77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:21:58.485118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49a640-5f5b-453c-bd1d-4540de9e4c77 service swift] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:21:58.493707 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1289/2584] 10.222.0.42 () {60 vars in 1384 bytes} [Mon Oct 24 13:21:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 24 13:22:00.568791 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80cab7-b617-4594-af23-c0fdf0de207c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:22:00.631349 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80cab7-b617-4594-af23-c0fdf0de207c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:22:00.636487 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:22:00.637106 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:22:00.637386 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:22:00.637659 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] PATH_INFO: `/v3/users/3f411b926c43411fa9b41fc40055d1e9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:22:00.638771 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f411b926c43411fa9b41fc40055d1e9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:22:00.640718 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:22:00.685962 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f411b926c43411fa9b41fc40055d1e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:00.698396 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f411b926c43411fa9b41fc40055d1e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:00.699066 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f411b926c43411fa9b41fc40055d1e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:00.706192 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f411b926c43411fa9b41fc40055d1e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:00.706856 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f411b926c43411fa9b41fc40055d1e9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:00.713004 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:22:00.721889 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1296/2585] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:22:00 2022] DELETE /identity/v3/users/3f411b926c43411fa9b41fc40055d1e9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 24 13:22:00.726510 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d70b77-d1cd-44eb-8d7c-38a0d7ecf69e None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:22:00.727001 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d70b77-d1cd-44eb-8d7c-38a0d7ecf69e None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:22:00.727118 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d70b77-d1cd-44eb-8d7c-38a0d7ecf69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:22:00.746154 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-75d70b77-d1cd-44eb-8d7c-38a0d7ecf69e None None] MFA Rules not processed for user `c47c715f72104fe1bd8f50179af11065`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:22:00.747398 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-75d70b77-d1cd-44eb-8d7c-38a0d7ecf69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:22:00.805348 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1290/2586] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 24 13:22:00.814877 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:22:00.816328 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:22:00.875021 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c6d892-6593-44fd-ac00-411cd82ac9ac admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:22:00.878674 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:22:00.879313 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:22:00.879599 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:22:00.879880 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:22:00.889872 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:22:00.891275 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:22:00.901278 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1297/2587] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:22:01.070175 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d70b77-d1cd-44eb-8d7c-38a0d7ecf69e None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:22:01.072785 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-75d70b77-d1cd-44eb-8d7c-38a0d7ecf69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:22:01.089976 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d70b77-d1cd-44eb-8d7c-38a0d7ecf69e None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:22:01.094876 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:22:01.095553 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:22:01.095727 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:22:01.096032 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] PATH_INFO: `/v3/projects/a827ef206ac2466b8b28f472bd1d4813` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:22:01.097136 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a827ef206ac2466b8b28f472bd1d4813)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:22:01.099099 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:22:01.147900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:22:01.151409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a827ef206ac2466b8b28f472bd1d4813', 'request_id': 'req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:01.158157 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a827ef206ac2466b8b28f472bd1d4813', 'request_id': 'req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:01.165705 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a827ef206ac2466b8b28f472bd1d4813', 'request_id': 'req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:01.176093 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] The token cache is being invalidate because project a827ef206ac2466b8b28f472bd1d4813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:22:01.176407 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:01.177035 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:22:01.179296 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1291/2588] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:22:01 2022] DELETE /identity/v3/projects/a827ef206ac2466b8b28f472bd1d4813 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:22:10.896861 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=28 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:22:11.172997 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:22:24.046662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:22:24.048198 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:22:24.060492 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58faae2-79f8-499a-ad7f-f18d84540f5c service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:22:24.109539 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:22:24.110139 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:22:24.110420 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:22:24.110690 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:22:24.112476 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:22:24.183553 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:22:24.185062 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:22:24.196458 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1298/2589] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:22:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:22:25.376195 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:22:25.377687 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:22:25.390541 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ac123-18dc-41ee-a2e9-b1f9edfdb32e admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:22:25.394038 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306252a-c37b-4973-8128-07a65f0b7ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:22:25.394634 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306252a-c37b-4973-8128-07a65f0b7ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:22:25.394911 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306252a-c37b-4973-8128-07a65f0b7ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:22:25.395216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306252a-c37b-4973-8128-07a65f0b7ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:22:25.396900 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-2306252a-c37b-4973-8128-07a65f0b7ba0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:22:25.410211 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306252a-c37b-4973-8128-07a65f0b7ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:22:25.411664 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306252a-c37b-4973-8128-07a65f0b7ba0 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:22:25.423222 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1292/2590] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:22:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:22:34.191307 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=54 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:22:35.418234 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=6 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:23:30.675484 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:30.677261 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:30.691918 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac2e4-43e5-4433-add8-184245a245b5 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:30.698982 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:30.699768 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:30.700115 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:30.700459 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:30.709178 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:30.725026 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:30.732047 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:30.750981 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1299/2591] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:31.908836 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d018d3-f31a-46fc-a42b-d0a2340e22f6 None None] REQUEST_METHOD: `POST` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:31.909315 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d018d3-f31a-46fc-a42b-d0a2340e22f6 None None] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:31.909420 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d018d3-f31a-46fc-a42b-d0a2340e22f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:31.929223 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.auth.core [None req-d9d018d3-f31a-46fc-a42b-d0a2340e22f6 None None] MFA Rules not processed for user `1fe85edb1188403d84072d19128e07d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81614) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:23:31.930422 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d9d018d3-f31a-46fc-a42b-d0a2340e22f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:31.984802 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1293/2592] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:23:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:23:32.294019 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:32.303265 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37bc3e-abf0-44e6-85e0-9d6c015bda6a service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:32.306834 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:32.307462 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:32.307744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:32.308018 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:32.309705 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:32.319238 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:32.321159 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:32.329083 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1300/2593] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:32.778004 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d018d3-f31a-46fc-a42b-d0a2340e22f6 None None] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:32.779522 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-d9d018d3-f31a-46fc-a42b-d0a2340e22f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:32.852409 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d018d3-f31a-46fc-a42b-d0a2340e22f6 None None] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:32.857305 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:32.857906 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:32.858186 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:32.858457 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:32.860245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:32.872048 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:32.873609 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:32.883274 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1294/2594] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:34.814762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:34.825152 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5091f3-d672-41cd-ab13-9c468ec8418f service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:34.830163 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:34.830899 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:34.831312 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:34.831646 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:34.833423 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:34.844833 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:34.846289 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:34.855846 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1301/2595] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:42.879141 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=57 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:23:44.851199 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=18 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:23:45.901917 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:45.903454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:45.920077 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633bc9db-bde8-4492-a195-efe7b3b3ac7d service placement] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:45.925839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:45.926454 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:45.926735 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:45.927011 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:45.928790 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:45.942722 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:45.944544 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:45.956941 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1295/2596] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 24 13:23:49.817371 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:23:49.832219 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f12a4-2cdd-4059-8d01-4d305cb811c1 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:23:49.838306 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2dff78b5eadc42608afbf2b14dd07e7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:23:49.838926 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:23:49.839243 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:23:49.839520 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:23:49.841301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:23:49.855154 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:23:49.856770 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:23:49.868534 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1302/2597] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:23:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:23:55.952116 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=6 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:23:59.863344 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:24:12.180442 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:24:12.181867 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:24:12.195781 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6b768-331e-43a3-9025-04598a9bc533 service nova] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:24:12.200719 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:24:12.201316 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:24:12.201597 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:24:12.201865 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:24:12.203572 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:24:12.221990 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:24:12.223642 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:24:12.236428 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1296/2598] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:24:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:24:15.226934 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:24:15.228422 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:24:15.301603 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3dda6-249e-4e91-bab1-bd2885891e04 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:24:15.306259 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:24:15.306890 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:24:15.307211 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:24:15.307524 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] PATH_INFO: `/v3/users/af98e8f344914ae59852d82d176b4dc2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:24:15.308652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af98e8f344914ae59852d82d176b4dc2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:24:15.310592 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:24:15.360867 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af98e8f344914ae59852d82d176b4dc2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.375744 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af98e8f344914ae59852d82d176b4dc2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.376489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af98e8f344914ae59852d82d176b4dc2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.385682 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af98e8f344914ae59852d82d176b4dc2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.386333 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af98e8f344914ae59852d82d176b4dc2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.400810 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1303/2599] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:24:15 2022] DELETE /identity/v3/users/af98e8f344914ae59852d82d176b4dc2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:24:15.581343 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:24:15.592013 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079cd1f1-bf4c-4907-a9ff-643108571a2c service placement] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:24:15.651108 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:24:15.651759 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:24:15.652044 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:24:15.652324 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] PATH_INFO: `/v3/projects/f0bee9967bdb41eca0acff51d5e105e9` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:24:15.653450 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0bee9967bdb41eca0acff51d5e105e9)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:24:15.655404 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:24:15.704511 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0bee9967bdb41eca0acff51d5e105e9', 'request_id': 'req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.711255 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0bee9967bdb41eca0acff51d5e105e9', 'request_id': 'req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.718440 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0bee9967bdb41eca0acff51d5e105e9', 'request_id': 'req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.728645 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] The token cache is being invalidate because project f0bee9967bdb41eca0acff51d5e105e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:24:15.728936 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.729561 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.731080 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1297/2600] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:24:15 2022] DELETE /identity/v3/projects/f0bee9967bdb41eca0acff51d5e105e9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:24:15.735079 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:24:15.736489 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:24:15.750528 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf472e-22f3-4b08-a674-fdb3dd5cba43 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:24:15.807025 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:24:15.807762 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:24:15.808048 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:24:15.808325 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] PATH_INFO: `/v3/users/0592dbfdf4724e179e1be25c121b2c51` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:24:15.809494 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0592dbfdf4724e179e1be25c121b2c51)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:24:15.811473 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:24:15.834255 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:24:15.859820 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0592dbfdf4724e179e1be25c121b2c51'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.872652 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0592dbfdf4724e179e1be25c121b2c51'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.873345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0592dbfdf4724e179e1be25c121b2c51'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.879763 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0592dbfdf4724e179e1be25c121b2c51'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.880569 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-21be20dd-6177-4444-89ee-8736e5124260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0592dbfdf4724e179e1be25c121b2c51'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:15.892929 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1304/2601] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:24:15 2022] DELETE /identity/v3/users/0592dbfdf4724e179e1be25c121b2c51 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:24:16.069105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:24:16.069105 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:24:16.082405 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb0dce-3ebb-4788-b754-c10869d1a6e8 admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:24:16.140064 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5347970-8026-463d-8226-64266119df7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:24:16.140689 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5347970-8026-463d-8226-64266119df7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:24:16.140973 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5347970-8026-463d-8226-64266119df7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:24:16.141245 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5347970-8026-463d-8226-64266119df7a admin admin] PATH_INFO: `/v3/projects/c7129d5827e94d7da7452b7803b1b147` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:24:16.142338 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5347970-8026-463d-8226-64266119df7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7129d5827e94d7da7452b7803b1b147)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:24:16.144254 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5347970-8026-463d-8226-64266119df7a admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:24:16.164839 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:24:16.192263 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a5347970-8026-463d-8226-64266119df7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7129d5827e94d7da7452b7803b1b147', 'request_id': 'req-a5347970-8026-463d-8226-64266119df7a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:16.198434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a5347970-8026-463d-8226-64266119df7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7129d5827e94d7da7452b7803b1b147', 'request_id': 'req-a5347970-8026-463d-8226-64266119df7a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:16.205731 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a5347970-8026-463d-8226-64266119df7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7129d5827e94d7da7452b7803b1b147', 'request_id': 'req-a5347970-8026-463d-8226-64266119df7a'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:16.216703 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a5347970-8026-463d-8226-64266119df7a admin admin] The token cache is being invalidate because project c7129d5827e94d7da7452b7803b1b147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:24:16.216993 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a5347970-8026-463d-8226-64266119df7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:16.217614 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-a5347970-8026-463d-8226-64266119df7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:24:16.219090 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1298/2602] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:24:16 2022] DELETE /identity/v3/projects/c7129d5827e94d7da7452b7803b1b147 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:24:25.889451 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:24:26.213762 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:23.063843 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829de180-96a9-4dbb-9273-38f7350bee81 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:23.064397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829de180-96a9-4dbb-9273-38f7350bee81 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:23.064397 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829de180-96a9-4dbb-9273-38f7350bee81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:23.085232 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-829de180-96a9-4dbb-9273-38f7350bee81 None None] MFA Rules not processed for user `1fe85edb1188403d84072d19128e07d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 24 13:25:23.086445 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-829de180-96a9-4dbb-9273-38f7350bee81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:23.143301 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1305/2603] 10.222.0.42 () {58 vars in 958 bytes} [Mon Oct 24 13:25:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 24 13:25:33.136118 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=25 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:25:36.200386 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5347970-8026-463d-8226-64266119df7a admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:25:36.201835 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-a5347970-8026-463d-8226-64266119df7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:36.273258 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5347970-8026-463d-8226-64266119df7a admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:25:36.279121 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:25:36.279728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:25:36.280008 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:25:36.280284 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:25:36.281979 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:25:36.293336 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:25:36.294952 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:25:36.304942 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1299/2604] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:25:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:25:46.300574 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=32 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:07.467821 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829de180-96a9-4dbb-9273-38f7350bee81 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:07.469247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-829de180-96a9-4dbb-9273-38f7350bee81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:07.532180 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829de180-96a9-4dbb-9273-38f7350bee81 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:07.535747 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:07.536350 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:07.536634 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:07.536911 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:07.538637 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:07.549495 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:07.550908 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:07.559665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1306/2605] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:17.555777 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=26 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:20.987716 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:20.989135 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:21.000768 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b628d3-eee1-4dd8-a4fd-ea8f9d2ea6c6 service placement] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:21.004353 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c414b04fe0544e3a938661bc957c8bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:21.004887 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:21.005168 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:21.005443 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:21.007133 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:21.072503 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:21.073986 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:21.083910 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1300/2606] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:27:27.291521 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:27.293085 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:27.307759 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528dbaf-9093-47b8-a391-d9b84249c5e4 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:27.312613 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:27.313247 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:27.313529 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:27.313804 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] PATH_INFO: `/v3/users/1fe85edb1188403d84072d19128e07d5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:27.314923 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fe85edb1188403d84072d19128e07d5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:27.317007 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:27.374481 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fe85edb1188403d84072d19128e07d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.386662 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fe85edb1188403d84072d19128e07d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.387379 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fe85edb1188403d84072d19128e07d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.395866 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fe85edb1188403d84072d19128e07d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.396868 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fe85edb1188403d84072d19128e07d5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.413526 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1307/2607] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:27 2022] DELETE /identity/v3/users/1fe85edb1188403d84072d19128e07d5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:27.591280 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:27.603753 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c8761-55cf-4e25-8a89-4074751ae0e5 service neutron] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:27.662728 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:27.663394 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:27.663677 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:27.663956 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] PATH_INFO: `/v3/projects/e5f4729aa5f44fe08096bed3438e8f7e` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:27.665069 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5f4729aa5f44fe08096bed3438e8f7e)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:27.666969 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:27.718185 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5f4729aa5f44fe08096bed3438e8f7e', 'request_id': 'req-02242fd8-39b2-428f-9c32-046706ca692d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.724418 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5f4729aa5f44fe08096bed3438e8f7e', 'request_id': 'req-02242fd8-39b2-428f-9c32-046706ca692d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.731290 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5f4729aa5f44fe08096bed3438e8f7e', 'request_id': 'req-02242fd8-39b2-428f-9c32-046706ca692d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.741925 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] The token cache is being invalidate because project e5f4729aa5f44fe08096bed3438e8f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:27.742216 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.742862 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.744376 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1301/2608] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:27 2022] DELETE /identity/v3/projects/e5f4729aa5f44fe08096bed3438e8f7e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:27.748969 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:27.750455 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:27.763612 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c3384b-aab8-4617-9b1e-b38731769b69 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:27.819005 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:27.819665 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:27.819951 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:27.820227 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] PATH_INFO: `/v3/users/8e513f79ed7b4a5292dd798e1776465a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:27.821354 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e513f79ed7b4a5292dd798e1776465a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:27.823317 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:27.869894 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e513f79ed7b4a5292dd798e1776465a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.882570 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e513f79ed7b4a5292dd798e1776465a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.883260 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e513f79ed7b4a5292dd798e1776465a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.891394 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e513f79ed7b4a5292dd798e1776465a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.892013 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e513f79ed7b4a5292dd798e1776465a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:27.894032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:27.906457 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1308/2609] 10.222.0.42 () {58 vars in 1224 bytes} [Mon Oct 24 13:27:27 2022] DELETE /identity/v3/users/8e513f79ed7b4a5292dd798e1776465a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:28.080798 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] Authenticating user token {{(pid=81614) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:27:28.082224 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.fernet_utils [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81614) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:27:28.095525 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02242fd8-39b2-428f-9c32-046706ca692d admin admin] Validating token access rules against request {{(pid=81614) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:27:28.117492 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:28.157160 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47c715f72104fe1bd8f50179af11065', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f3929898cf764cd6a05e8662e83d95e0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81614) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:27:28.157820 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:27:28.158107 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] SCRIPT_NAME: `/identity` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:27:28.158383 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] PATH_INFO: `/v3/projects/f4c3f792e1ae485fa9730e731b795550` {{(pid=81614) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:27:28.159531 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c3f792e1ae485fa9730e731b795550)` {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:27:28.161434 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] RBAC: Authorization granted {{(pid=81614) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:27:28.207459 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c3f792e1ae485fa9730e731b795550', 'request_id': 'req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:28.213519 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c3f792e1ae485fa9730e731b795550', 'request_id': 'req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:28.221611 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c3f792e1ae485fa9730e731b795550', 'request_id': 'req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d'} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:28.231841 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] The token cache is being invalidate because project f4c3f792e1ae485fa9730e731b795550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81614) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 24 13:27:28.232127 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:28.233373 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG keystone.notifications [None req-16f572f1-9b1d-4f23-ae8e-b02f78a5089d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81614) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 24 13:27:28.234912 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: [pid: 81614|app: 0|req: 1302/2610] 10.222.0.42 () {58 vars in 1233 bytes} [Mon Oct 24 13:27:28 2022] DELETE /identity/v3/projects/f4c3f792e1ae485fa9730e731b795550 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 24 13:27:37.905700 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:27:38.228825 dv-cinder-862489-1-iscsi devstack@keystone.service[81614]: DEBUG dbcounter [-] [81614] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81614) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 24 13:30:41.188345 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 24 13:30:41.189764 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:41.326432 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b57e8-5b75-48f9-a8f5-9f1e37832156 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 24 13:30:41.384953 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca3ec6-a8cc-40d1-8e66-0c063f7ad6ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '81c4bb19721146c7992d8ea11cd165de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '00530732ebe84dababafb9f399452940', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 24 13:30:41.385565 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3ec6-a8cc-40d1-8e66-0c063f7ad6ba service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 24 13:30:41.385848 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3ec6-a8cc-40d1-8e66-0c063f7ad6ba service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 24 13:30:41.386120 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca3ec6-a8cc-40d1-8e66-0c063f7ad6ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 24 13:30:41.387858 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-36ca3ec6-a8cc-40d1-8e66-0c063f7ad6ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 24 13:30:41.399032 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3ec6-a8cc-40d1-8e66-0c063f7ad6ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 24 13:30:41.400795 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca3ec6-a8cc-40d1-8e66-0c063f7ad6ba service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 24 13:30:41.410916 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1309/2611] 10.222.0.42 () {60 vars in 1365 bytes} [Mon Oct 24 13:30:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 24 13:30:51.406663 dv-cinder-862489-1-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}